Systems and methods for automated distortion correction and/or co-registration of three-dimensional images using artificial landmarks along bones

ABSTRACT

Presented herein are systems and methods for registering one or more images of one or more subjects based on the automated generation of artificial landmarks. An artificial landmark is a point within an image that is associated with a specific physical location of the imaged region. The artificial landmarks are generated in an automated and robust fashion along the bones of a subject&#39;s skeleton that are represented in the image (e.g. graphically). The automatically generated artificial landmarks are used to correct distortion in a single image or to correct distortion in and/or co-register multiple images of a series of images (e.g. recorded at different time points). The artificial landmark generation approach described herein thereby facilitates analysis of images used, for example, for monitoring the progression of diseases such as pulmonary diseases.

CROSS REFERENCE SECTION

This application is a National Stage Application under 35 U.S.C. 371 ofPCT application number PCT/US2018/037708 designating the United Statesand filed Jun. 15, 2018; which claims priority to U.S. ProvisionalApplication Ser. No. 62/520,926, filed Jun. 16, 2017, and entitled“SYSTEMS AND METHODS FOR AUTOMATED DISTORTION CORRECTION AND/ORCO-REGISTRATION OF THREE-DIMENSIONAL IMAGES USING ARTIFICIAL LANDMARKSALONG BONES”, each of which are hereby incorporated by reference intheir entireties.

FIELD OF THE INVENTION

This invention relates generally to methods and systems of imageanalysis. More particularly, in certain embodiments, the inventionrelates to automated distortion correction and/orregistration/co-registration of one or more subject images (e.g. one ormore images comprising portions of an axial skeleton of the subject),e.g., captured with a computed tomography (CT) scanner.

BACKGROUND OF THE INVENTION

In vivo imaging of small mammals is performed by a large community ofinvestigators in various fields, e.g., oncology, infectious disease, anddrug discovery. There is a wide array of technologies directed to invivo imaging of mammals—for example, bioluminescence, fluorescence,tomography, and multimodal imaging technologies.

In vivo imaging often involves the use of markers such as fluorescentprobes, radionuclides and the like, for non-invasive spatiotemporalvisualization of structures and/or biological phenomena inside a liveanimal. For example, fluorescence molecular tomography (FMT) involves invivo imaging of mammals for quantitative analysis of administered and/orendogenous probes. In vivo micro Computer Tomography (microCT) imaging,is an x-ray-based technology that scans and produces images of tissues,organs, and non-organic structures with high resolution. MicroCT hasevolved quickly, requiring low dose scanning and fast imaging protocolsto facilitate multimodal applications and enable longitudinalexperimental models. Multimodal imaging involves the fusion of imagesobtained in different ways, for example, by combining FMT, positronemission tomography (PET), magnetic resonance imaging (MRI), CT,microCT, and/or single photon emission computed tomography (SPECT)imaging data.

Analysis of images, such as microCT images, often involves automatedimage processing (performed, e.g. by various image analysis software) inorder to, for example, identify regions of interest within images (e.g.regions corresponding to particular bones, tissue types, organs). Otherimage processing may also be used in order to facilitate identificationof abnormal features (e.g. tissue morphology, masses such as tumors)within a single image of a subject, and/or identification of changes bycomparing multiple images of a subject recorded at different times. Incertain cases, comparison of images of different subjects is also usefulfor identification of abnormal features, and may be facilitated throughthe use of image processing.

Image analysis methods that facilitate comparison of multiple imagesrecorded at different time points is particularly relevant formonitoring disease progression in a subject (e.g. a small mammal, suchas a mouse, rat, guinea pig, or rabbit). For example, changes in tissuemorphology, including tumor growth, edema, and fibrosis, can be deducedvia analysis of multiple images collected over a period of time.Monitoring disease progression in this manner is relevant for diagnosingsubject status, gauging treatment effectiveness, and investigatinggeneral aspects of different diseases. Monitoring diseases via imageanalysis thus can provide new insights relevant for diagnosingconditions, applying appropriate therapies, and developing new treatmentprocedures.

This is especially true for progressive diseases, such as pulmonarydiseases. Pulmonary diseases affect the lungs, and include diseases suchas asthma, infections such as influenza and pneumonia, as well as lungcancer. Monitoring the progression of pulmonary diseases involvesrecording changes in characteristics of the lungs (e.g. tumor growth,edema, fibrosis, a downward expansion of lungs in respect to a subject'sribcage) over time. Imaging techniques, such as computed tomography(e.g. microCT) are used to record images of a region of a subjectcomprising the subject's lungs at different times. Comparing such imagestaken at different time points allows changes in the subject's lungs tobe observed over time.

Comparing different images taken at different times, however, is quitechallenging. In particular, variations in the pose and position of thesubject from one image to a next produces variations between imagesrecorded at different time points. For example, graphicalrepresentations of tissue (e.g. lung tissue, connective tissue, bones)within different images of a subject may be distorted, positioned,and/or rotated differently from image to image, depending on the poseand position of the subject during recording of a given image. Forexample, FIG. 1 shows a series of graphical representations of axialskeleton bones of a mouse. These are projections of a 3D mask, whereintensity of each pixel is the average intensity of mask voxels alongthe third dimension. Each projection corresponds to a differentthree-dimensional representation of a region of the mouse as itsposition is adjusted and an image is recorded at different times. As isevident in FIG. 1, both the distortion and position of the imagedskeleton differ from image to image, as a result of variations in boththe pose and location of the subject between the images. Such variationsmust be removed in order to allow for meaningful analysis and comparisonof a series of multiple images recorded over a period of time.

Thus, there is a need for systems and methods that provide forregistration of, including correction of distortions and orientationvariations between, multiple images of a subject or images of differentsubjects. Systems and methods that address this need would facilitateanalysis of multiple images of a subject that are recorded at differenttime points, as well as comparisons between images of differentsubjects. Such analysis is relevant for many applications, for example,monitoring disease progression (e.g. of relevance for pulmonarydiseases).

SUMMARY OF THE INVENTION

Presented herein are systems and methods for registering one or moreimages of one or more subjects based on the automated generation ofartificial landmarks. An artificial landmark is a point within an imagethat is associated with a specific physical location of the imagedregion. The artificial landmarks are generated in an automated androbust fashion along the bones of a subject's skeleton that arerepresented in the image (e.g. graphically). The automatically generatedartificial landmarks are used to correct distortion in a single image orto correct distortion in and/or co-register multiple images of a seriesof images (e.g. recorded at different time points). The artificiallandmark generation approach described herein thereby facilitatesanalysis of images used, for example, for monitoring the progression ofdiseases such as pulmonary diseases.

For example, transformations derived from the artificially-generatedbone landmarks can be used to correct distortion and/or co-registerregions of images representing soft tissue (e.g., lungs, heart, etc.).As used herein, “image registration” refers to distortion correction ofone or more images and/or co-registration of multiple images. Thegraphical representation of soft tissue being registered may be in thesame image(s) for which the bone registration is performed, or the softtissue may be in different images that are obtained at the same time asthe images used for bone registration (e.g., via a multi-modalityimaging system), or, if otherwise, with the subject in the same pose asin the corresponding bone registration images.

The use of landmarks along bones of a subject for registration (e.g.distortion correction and/or co-registration) is advantageous formonitoring progressive diseases since the configuration of a subject'sskeletal bones remains substantially constant over time, while themorphology of soft tissue (e.g. lungs, heart, etc.) may change inresponse to the disease.

In certain embodiments, the systems and methods for registration (e.g.distortion correction and/or co-registration) described herein areapplied to facilitate analysis of images representing a region of asubject comprising the subject's lungs. Such images comprise graphicalrepresentations of axial skeleton bones of the subject, such as ribbones, a backbone, and a breastbone. In certain embodiments, artificiallandmarks are automatically generated along each of a plurality of ribbones represented within the image. In certain embodiments, artificiallandmarks are additionally automatically generated along a breastbonerepresented within the image.

In certain embodiments, artificial landmarks along rib bones and/or abreastbone represented in a single image are used to correct distortionspresent in the image that result from (e.g. reflect) the particular posein which the subject was situated while the image was recorded. Inparticular, artificial landmarks within the image can be used todetermine a transformation [e.g. a linear transformation (e.g. a rigidtransformation (e.g. a translation, e.g. a rotation), an affinetransformation, e.g. any combination of one or more rigidtransformations and/or affine transformations); e.g. a nonlineartransformation (e.g. a regularized or smoothed extrapolation ofdistortion field, e.g. a thin-plate spline, e.g. a transformationcomprising an extrapolation operation and/or a smoothing operation);e.g. a combination of one or more linear transformations and one or morenonlinear transformations] that, when applied to the image, produces asymmetrical representation of the region of the subject. Suchsymmetrical, standardized images can readily be analyzed, interpreted,and/or compared with other images (e.g. previously recorded images, e.g.previously recorded images that were corrected for distortion similarly)in a direct fashion, without having to account for variations in pose ofthe subject.

In certain embodiments, artificial landmarks are used to co-registermultiple images of a series of images (e.g. recorded at different timepoints) of a region of a subject. In particular, in certain embodiments,by virtue of their association with specific physical locations of theimaged region, artificial landmarks of different images can be used todetermine a transformation that co-registers the images (e.g. aligns theplurality of images to each other). In certain embodiments, a baselineimage of a set of images is selected, to which the other images arealigned (e.g. for n images, each n−1 images may be transformed toco-register them with a selected, baseline image of the n images)). Incertain embodiments, the baseline image to which each image of a set ofimages is aligned is not selected from the set of images, and isinstead, for example, a pre-defined standard baseline image.

In certain embodiments, artificial landmarks are used to both correctdistortion and co-register multiple images of a series of images of aregion of the subject. In particular, for each image of a series ofimages, artificial landmarks of the image are used to determine atransformation that (i) corrects distortion in the image (e.g. resultingfrom pose variations of the subject) and (ii) aligns the image with theother images of the series. Accordingly, for each image, application ofthe determined transformation yields a symmetrized image that isco-registered with the other images of the series of images.

In particular, in certain embodiments, image registration (e.g.correcting distortion in one or more images, and/or co-registeringmultiple images) is accomplished by determining, for each image of oneor more images, a transformation that, when applied to the image,substantially optimizes the alignment of the set of artificial landmarkswithin the image with a set of target landmarks.

Depending on the positions of the target landmarks, the determinedtransformation that aligns the artificial landmarks in the image withthe target landmarks may correct distortions in the image (e.g. yield asymmetrized version of the image) and/or co-register the image with oneor more different images.

For example, in certain embodiments, symmetric target landmarks are usedfor image registration. In order to register a given image, atransformation is determined using artificial landmarks in the image andthe symmetric set of target landmarks. The determined transformation canbe applied to the image to generate a symmetrized version of the image,thereby correcting distortions. In certain embodiments, a single set oftarget landmarks is used for registration of multiple images, therebyproviding for co-registration of the images. This registration may beconducted for multiple images of a single subject (e.g., collected atdifferent time points) or of multiple subjects. If the single set oftarget landmarks used for registration of multiple images is symmetric,then both distortion correction and co-registration are accomplished.

In certain embodiments, target landmarks used for registration of aparticular image are derived from artificial landmarks within theparticular image. For example, correcting distortion in a single imagecomprises determining a symmetric set of target landmarks from theartificial landmarks in the image, and determining a transformation thatoptimizes the alignment of the set of artificial landmarks within theimage with a set of target landmarks. The determined transformation isapplied to the image, thereby producing a symmetric image.

In certain embodiments, target landmarks used for registering a secondimage are obtained from artificial landmarks of a first image. Forexample, in order to co-register the second image with the first image,the artificial landmarks of the first image may be used as targetlandmarks for registration of the second image. Accordingly, determininga transformation that aligns artificial landmarks in the second imagewith the target landmarks (which are the artificial landmarks of thefirst image) and applying the transformation to the second imageco-registers the second image with the first image.

In certain embodiments, target landmarks are determined from the firstimage, in order to, as described above, provide for distortioncorrection (e.g. symmetrization) of the first image. The set of targetlandmarks can be used for distortion correction and registration of thefirst image, as well as for distortion correction and registration ofone or more other images.

In certain embodiments, the approaches described herein are used fordistortion correction and/or co-registration of multiple images of asingle subject. In certain embodiments, the approaches described hereinare used for distortion correction and/or co-registration of images ofdifferent subjects.

An advantage of the artificial landmark generation approach describedherein is its robustness to artifacts, such as artifacts resulting fromerrors in automated identification (e.g. via segmentation) ofrepresentations of particular bones within the images, as well asvariations in image quality. In particular, in certain embodiments,artificial landmarks are determined from artificial objects (e.g. eachartificial object is determined as a center of mass of a correspondingartificial object) generated along bones by morphological and logicalimage processing operations that are automatically applied to the images(e.g. generation of distance interval masks, e.g. logical ANDoperations). Accordingly, in contrast to naturally occurring objects,such as specific bones, joints, and the like, artificial objectscorrespond to fractions of real bones, generated in silico, within animage via a series of specific morphological and logical imagingprocessing operations.

A surprising result is that generation of artificial objects asdescribed herein is more robust and tolerant of variations in imagequality than identification of natural objects. Identification ofspecific bones, joints, and the like within images via automatedsegmentation is prone to segmentation errors in certain images, such aslower quality images. Correction of segmentation errors may beaccomplished interactively, with the help of a user, but this is verytedious, in particular if there are tens of segmentation errors on eachimage among several images in a time series.

In certain embodiments, generation of artificial objects via the methodsand systems described herein obviates the need for extensive imagesegmentation. For example, artificial objects may be generated along ribbones within an image via the approach described herein without havingto separately identify individual rib bones, which is more challengingand error prone. Accordingly, artificial objects (and, in turn,artificial landmarks generated from artificial objects) are generated ina fully automated fashion, without user interaction. The process ofgenerating artificial objects, and, from them, artificial landmarks foruse in image registration, is robust to image artifacts, errors inautomated identification of particular bones, and can be used with lowerquality images than approaches that require accurate identification ofnatural objects.

Additionally, unlike natural objects such as specific joints and fullbones, the number and density of artificial objects, and, accordingly,landmarks derived from them, can be freely selected. The ability tocontrol the number and density of artificial objects in an image isadvantageous for determining transformations for image registration.

In certain embodiments, in order to ensure accurate registration ofimages, the systems and methods described herein include steps forensuring that generated artificial landmarks (and/or artificial objects)are correctly identified (e.g., the correspondence between a givenartificial landmark (or artificial object) and a particular physicallocation is correctly identified (e.g., via assignment of indexvalues)). For example, in certain cases, accurate identification ofartificial landmarks (and/or artificial objects) ensures that there areno unwanted artifacts among generated seed artificial objects and,similarly, that there are no unwanted artifacts among subsequentartificial objects generated from a seed artificial object.Additionally, correct identification of artificial landmarks and/orartificial objects ensures that pairs opposite rib bone partnerlandmarks are properly identified when determining a set of symmetrictarget landmarks for distortion correction. Lastly, correctidentification of artificial landmarks and/or artificial objects ensuresthat artificial landmarks are correctly matched to partner targetlandmarks for determining image registration transforms.

Accordingly, in certain embodiments, artificial landmarks (and/orartificial objects) are identified (e.g., via assignment of variousindex values) during the generation process, and the correctness of theidentification may be checked at subsequent steps when artificiallandmarks are used for image registration. For example, correctness ofidentification may be checked when determining pairs of opposite ribbone artificial landmarks for generation of target landmarks, as well aswhen matching artificial landmarks in an image to target landmarks forimage registration. In certain embodiments, if identification errors arefound, artificial landmarks (and/or artificial objects) that aremisidentified are removed from the set of artificial objects (and/orartificial landmarks) in an image, and not used for image registration.

In certain embodiments, the accuracy and robustness with whichartificial landmarks are generated from artificial objects is furtherimproved via outlier filtering steps that are applied to removeartificial objects (and/or artificial landmarks determined from suchartificial objects) that do not conform to specific criteria. Forexample, artificial objects may be filtered based on volume in order toremove very small artificial objects that result from e.g. noise andsmall artifacts in an image. In certain embodiments, criteria used foroutlier filtering are based on physical intuition regarding the regionof the subject that is imaged, such as the fact that a subject's ribcageis substantially symmetric. Such outlier filtering approaches allowartificial landmarks to be accurately generated with minimal userinteraction (e.g. without any user interaction) during the generationprocess.

Accordingly, by providing an approach for registration (e.g. distortioncorrection and/or co-registration) of 3-D images of a subject orsubjects based on automated generation of artificial landmarks, thesystems and methods described herein facilitate the analysis of diseaseprogression in a subject via comparison of tomographic images of thesubject. The approach described herein is relevant to images featuringbones of a subject's axial skeleton, including the ribcage, backbone,and breastbone. By providing a robust and automated approach fordistortion correction and image co-registration, the systems and methodsdescribed herein improve capabilities for monitoring diseaseprogression, particularly relevant for pulmonary diseases.

In one aspect, the invention is directed to a method for registration ofone or more 3-D images of a subject, the method comprising: (a)receiving, by a processor of a computing device, a 3-D image of thesubject (e.g. a tomographic image, e.g., an image obtained via CT,micro-CT, SPECT, x-ray, MR, ultrasound, and/or a combination of any ofthese), wherein the 3-D image comprises a graphical representation ofrib bones and a backbone; (b) identifying in the graphicalrepresentation [e.g., graphically isolating, e.g. automatically (e.g.via segmentation); e.g. via a user interaction], by the processor, (i)the backbone and (ii) ribcage bones comprising the rib bones [{e.g., bydetermination of a bone mask followed by separation of identified bonetissue into a first part corresponding to an identified backbone and asecond part corresponding to identified rib bones} {e.g., bydetermination of a bone mask followed by separation of identified bonetissue into a first part corresponding to an identified backbone and asecond part corresponding to identified ribcage bones (e.g. comprisingrib bones together with a breastbone), optionally followed by separationof the identified ribcage bones into an identified breastbone and a partcorresponding to the remaining bones (i.e. the rib bones)}]; (c)automatically generating, by the processor, a plurality of seed rib boneartificial objects, each seed rib bone artificial object being within afirst distance interval from the backbone in the graphicalrepresentation (e.g. (i) for each point of the seed rib bone artificialobject, a shortest distance from the point to a surface of theidentified backbone is within the first distance interval; e.g. (ii) foreach point of the seed rib bone artificial object, a shortest distancefrom the point to a centerline of the identified backbone is within thefirst distance interval) and corresponding to a region of a rib bone inthe graphical representation (e.g., wherein each seed rib boneartificial object is assigned a left-right index and a rib numberindex); (d) for each automatically generated seed rib bone artificialobject, automatically generating, by the processor, a plurality ofassociated subsequent rib bone artificial objects, thereby creating aset of artificial objects (e.g. the set comprising the seed andsubsequent rib bone artificial objects) within image; (e) automaticallyperforming, by the processor, registration of one or more images of thesubject using the set of artificial objects within the image (e.g.,using a set of artificial landmarks, each landmark determined based onits corresponding artificial object within the image, e.g. wherein eachartificial landmark of the set of artificial landmarks is calculated asa center of mass of each artificial object of the set of artificialobjects).

In certain embodiments, identifying the back bone and rib bonescomprises determining a set of one or more bone masks to identifyspecific bones in the graphical representation. For example, determiningthe set of bone masks may be conducted in multiple steps—e.g., a firststep to identify all of the bones in the graphical representation, asecond step to identify the backbone, and a third step to identify(undifferentiated) ribcage bones comprising the rib bones. In certainembodiments, the identified ribcage bones include rib bones togetherwith a breastbone. In certain embodiments, a fourth step is performedthat separates the identified ribcage bones into (i) an identifiedbreastbone and (ii) remaining bones corresponding to identified ribbones. In certain embodiments, it is not necessary to separatelyidentify the breastbone and rib bones, and the identified ribcage bonescorrespond to an undifferentiated collection of rib bones together withthe breast bone. In certain embodiments, the breast bone and rib bonesare separately identified, and operations described herein as beingperformed on identified ribcage bones and/or a mask corresponding toidentified ribcage bones are performed on identified rib bones and/or amask corresponding to identified rib bones. In certain embodiments, itis not necessary to separately identify individual rib bones (thoughthis may be performed in other embodiments), but rather, anundifferentiated collection of rib bones is identified.

In certain embodiments, the step of automatically generating theplurality of seed rib bone artificial objects comprises: determining abackbone distance map comprising intensity (e.g. numeric) values at eachof a plurality of points in three dimensions, each of the intensityvalues of the backbone distance map corresponding to a distance from thegiven point in 3-D space to the identified backbone in the graphicalrepresentation (e.g. (i) a shortest distance from the given point to apoint of the identified backbone; e.g. (ii) a shortest distance from thegiven point to a centerline of the identified backbone); generating, bythe processor, a backbone distance interval mask from the backbonedistance map, the backbone distance interval mask corresponding toregions of the image that are within the first distance interval fromthe identified backbone (e.g. the distance interval mask is a binarymask, wherein points having a distance to the identified backbone withina distance interval (e.g. the first distance interval) are assigned afirst value (e.g. a numeric 1, e.g. a Boolean true), and other pointsare assigned a second value (e.g. a numeric 0, e.g. a Boolean false));and applying, by the processor, an AND operation (e.g. a point-wise ANDoperation) between the backbone distance interval mask and a maskcorresponding to the identified ribcage bones (e.g., the identified ribbones; e.g. a ribcage bones mask, e.g., a rib bones mask), therebyidentifying a plurality of regions of the identified ribcage bones thatare within the first distance interval from the identified backbone.

In certain embodiments, step (c) comprises, for each of the plurality ofseed rib bone artificial objects, associating with the seed rib boneartificial object (i) a left-right index (e.g. based on an x-coordinatedetermined from one or more points of the seed rib bone artificialobject) and (ii) a rib number index (e.g. based on a z-coordinatedetermined from one or more points of the seed rib bone artificialobject).

In certain embodiments, associating the seed rib bone artificial objectwith a left-right index comprises, as a preliminary step of applying apreliminary rotation and translation transformation; and, subsequently,determining the left-right index (e.g. based on an x-coordinatedetermined from one or more points of the seed rib bone artificialobject). In certain embodiments, the preliminary rotation andtranslation transformation is determined using principal axes of theidentified backbone and/or principal axes of an identified breastbone inthe graphical representation.

In certain embodiments, for each automatically generated seed rib boneartificial object, the plurality of associated subsequent rib boneartificial objects comprises an equidistant chain of rib bone artificialobjects.

In certain embodiments, the 3-D image comprises a graphicalrepresentation of rib bones, a backbone, and a breastbone, and each ribbone artificial object is at least a predefined first threshold distancefrom an identified breastbone in the graphical representation (e.g., foreach point of each rib bone artificial object, a shortest distance fromthe point to the identified breastbone is at least the first thresholddistance, e.g., for each point of each rib bone artificial object, ashortest distance from the point to a centerline of the identifiedbreastbone is at least the first threshold distance).

In certain embodiments, the method comprises automatically identifying,by the processor, one or more dangerous regions of the imagecorresponding to regions in which a distance between a first and secondrib bone in the graphical representation is below a predefined secondthreshold distance (e.g., automatically identifying the one or moredangerous regions of the image by applying one or more morphologicaloperations (e.g. including a morphological closing operation) and one ormore logical operations to a mask corresponding to the identifiedribcage bones), wherein the method comprises ensuring that eachgenerated rib bone artificial object is sufficiently far from anyidentified dangerous region within the image (e.g., is at least apredefined third threshold distance from the nearest identifieddangerous region, e.g., for each point of each rib bone artificialobject, a shortest distance from the point to the identified dangerousregion is at least the third threshold distance).

In certain embodiments, automatically generating, by the processor, theplurality of subsequent rib bone artificial objects associated with arespective seed rib bone artificial object comprises generating a chainof rib bone artificial objects along a rib bone by beginning with therespective seed rib bone artificial object and, in a stepwise fashion,generating new subsequent rib bone artificial objects, each newlygenerated rib bone artificial object proceeding outwards (e.g. apredefined distance from the previously generated rib bone artificialobject), away from the identified backbone, along the rib bone.

In certain embodiments, the 3-D image comprises a graphicalrepresentation of a plurality of rib bones, a backbone, and abreastbone, and generating the chain of artificial rib bone associatedwith the respective seed rib bone artificial object comprises, for atleast one newly generated rib bone artificial object of the chain of ribbone artificial objects: determining whether the newly generated ribbone artificial object is within a predetermined first thresholddistance from an identified breastbone in the graphical representation;and responsive to determining that the newly generated rib boneartificial object is within the predetermined first threshold distancefrom the identified breastbone, terminating generation of subsequentartificial objects associated with the respective seed rib boneartificial object.

In certain embodiments, the method comprises identifying, by theprocessor, one or more dangerous regions of the image corresponding toregions in which a distance between a first and second rib bone in thegraphical representation is below a predefined second thresholddistance, and wherein generating the chain of rib bone artificialobjects associated with the respective seed rib bone artificial objectcomprises, for at least one newly generated rib bone artificial objectof the chain of rib bone artificial objects: determining whether thenewly generated rib bone artificial object is within a predeterminedthird threshold distance from an identified dangerous region of theimage; and responsive to determining that the newly generated rib boneartificial object is within the third predetermined threshold distancefrom the identified dangerous region of the image, terminatinggeneration of subsequent rib bone artificial objects associated with therespective seed rib bone artificial object.

In certain embodiments, automatically identifying the one or moredangerous regions comprises applying one or more morphological closingoperations to a mask corresponding to identified ribcage bones.

In certain embodiments, the 3-D image comprises a graphicalrepresentation of at least a portion of a breastbone of the subject, theportion comprising a lower (e.g. tail-sided) end of the breastbone, andwherein the method comprises automatically generating, by the processor,a series of artificial objects along the breastbone by: (f) identifyingin the graphical representation [e.g. graphically isolating, e.g.,automatically (e.g., via segmentation), e.g. via a user interaction], bythe processor, the breastbone [{e.g., by determination of a bone maskfollowed by separation of identified bones into the backbone, abreastbone, and remaining bones (i.e., the rib bones)} {e.g., bydetermination of a bone mask followed by separation of identified bonetissue into a first part corresponding to an identified backbone and asecond part corresponding to identified ribcage bones (e.g. comprisingrib bones together with a breastbone), followed by separation of theidentified ribcage bones into an identified breastbone and a partcorresponding to the remaining bones (i.e., the rib bones)}] and thelower end of the breastbone (e.g. by determining a minimal z-coordinateof a breastbone mask corresponding to the identified breastbone); (g)automatically generating, by the processor, a seed breastbone artificialobject, the seed breastbone artificial object corresponding to a regionof the identified breastbone that is within a second distance intervalfrom the lower end of the breastbone (e.g. (i) for each point of theseed breastbone artificial object, a shortest distance from the point toa surface of the identified lower end of the breastbone is within thesecond distance interval; e.g. e.g. (ii) for each point of the seedbreastbone artificial object, a distance (e.g. a Euclidean distance,e.g. a distance along a particular coordinate (e.g. a z-coordinate))from the point to a point corresponding to the identified lower end ofthe breastbone is within the second distance interval); and (h)beginning with the seed breastbone artificial object, automaticallygenerating, by the processor, a plurality of associated breastboneartificial objects along the identified breastbone, the plurality ofbreastbone artificial objects included within the set of artificialobjects within the image (e.g. thereby creating a set of artificialobjects further comprising the generated breastbone artificial objects).

In certain embodiments, automatically generating, by the processor, theseed breastbone artificial object comprises: determining a breastbonelower end distance map comprising intensity (e.g. numeric) values ateach of a plurality of points in three dimensions, each of the intensityvalues of the backbone distance map corresponding to a distance from agiven point in 3-D space to the identified lower end of the breastbonein the graphical representation [e.g. (i) a shortest distance from thegiven point to a surface of the identified lower end of the breastbone;e.g. (ii) a distance (e.g. a Euclidean distance, e.g. a distance along aparticular coordinate (e.g. a z-coordinate)] from the given point to apoint corresponding the identified lower end of the breastbone);generating, by the processor, a breastbone lower end distance intervalmask from the breastbone lower end distance map, the breastbone lowerend distance interval mask corresponding to regions of the image thatare within the second distance interval from the lower end of thebreastbone (e.g. the breastbone lower end distance interval mask is abinary mask, wherein points having a distance to the lower end of thebreastbone within a predefined distance interval (e.g. the seconddistance interval) are assigned a first value (e.g. a numeric 1, e.g. aBoolean true), and other points are assigned a second value (e.g. anumeric 0, e.g. a Boolean false)); and applying, by the processor, anAND operation (e.g. a point-wise AND operation) between the breastbonelower end distance interval mask and a mask corresponding to theidentified breastbone, thereby identifying a region of the identifiedbreastbone that is within the second distance interval from the lowerend of the breastbone.

In certain embodiments, the plurality of breastbone artificial comprisesan equidistant chain of breastbone artificial objects along thebreastbone.

In certain embodiments, automatically generating the plurality ofbreastbone artificial objects comprises generating a chain of breastboneartificial object by beginning with the seed breastbone artificialobject and, in a stepwise fashion, generating new subsequent breastboneartificial objects, each newly generated breastbone artificial objectproceeding upwards (e.g. a predefined distance from the previouslygenerated breastbone artificial object), away from the identified lowerend of the breastbone.

In certain embodiments, the portion of the breastbone in the graphicalrepresentation comprises an upper end of the breastbone, the methodcomprises identifying in the graphical representation, by the processor,the upper end of the breastbone, and generating the chain of breastboneartificial objects comprises, for at least one newly generatedbreastbone artificial object: determining whether the newly generatedbreastbone artificial object reaches or is within a predeterminedthreshold distance from the identified upper end of the breastbone; andresponsive to determining that the newly generated breastbone artificialobject reaches or is within the predetermined threshold distance fromthe identified upper end of the breastbone, terminating generation ofsubsequent breastbone artificial objects.

In certain embodiments, each automatically generated artificial objectof the set of artificial objects within the image (e.g. each rib boneartificial object and/or each breastbone artificial object) is confirmedto have at least a predefined threshold volume.

In certain embodiments, automatically generating the plurality of seedrib bone artificial object comprises: generating, by the processor, aset of prospective seed rib bone artificial objects corresponding toplurality of regions of the rib bones in the graphical representationthat are within a distance interval from the backbone in the graphicalrepresentation; applying, by the processor, a volume filter to the setof prospective seed rib bone artificial objects, wherein application ofthe volume filter eliminates, from the set, those artificial objectsdetermined, by the processor, to represent a volume below a predefinedthreshold volume; and following application of the volume filter,selecting, by the processor, each seed rib bone artificial object fromthe set of prospective seed rib bone artificial objects.

In certain embodiments, the rib bones in the graphical representationinclude a plurality of [e.g., corresponding to a number of rib bones ona given side of the subject (e.g., a known number of rib bones)] pairsof opposite rib bones (e.g. each pair of opposite rib bones comprising aright rib bone and a corresponding left rib bone), each rib boneartificial object of a portion of the plurality of rib bone artificialobjects belongs to one of a plurality of pairs of rib bone artificialobjects, each pair comprising a first rib bone artificial objectassociated (e.g. based on an associated left-right index of rib boneartificial object) with a first rib bone (e.g. a right rib bone) of apair of opposite rib bones and a second rib bone artificial objectassociated (e.g. based on an associated left-right index of rib boneartificial object) with a second rib bone (e.g. a left rib bone) of thepair of opposite rib bones, and for each pair of rib bone artificialobjects, (i) a difference between a volume represented by the firstobject of the pair and a volume represented by the second object of thepair is below a predefined threshold volume difference, and/or (ii) adifference between a height [e.g., a thickness of the object in az-direction; e.g., distance of the object along the backbone (e.g.,determined as an average z-coordinate of the object)] of the firstobject of the pair and a height of the second object of the pair isbelow a predefined threshold height difference.

In certain embodiments, the of rib bones in the graphical representationinclude one or more pairs of opposite rib bones (e.g., each pair ofopposite rib bones comprising a right rib bone and a corresponding leftrib bone), and automatically generating the plurality of seed rib boneartificial object comprises: generating, by the processor, a set ofprospective seed rib bone artificial objects corresponding to pluralityof regions of the rib bones in the graphical representation that arewithin the first distance interval from the backbone; automaticallyidentifying, by the processor, one or more pairs of prospective seed ribbone artificial objects, each pair comprising a first seed rib boneartificial object of a first rib bone of the pair of rib bones andcorresponding second seed rib bone artificial object of the opposite ribbone (e.g. based on z-coordinates of each seed rib bone artificialobject; e.g. based on an associated rib number index of each seed ribbone artificial object; e.g. based on an associated left-right index ofeach seed rib bone artificial object); applying, by the processor, apair comparison filter to the set of prospective seed rib boneartificial objects, wherein application of the pair comparison filtereliminates, from the set, pairs of artificial objects for which (i) adifference between a volume represented by the first object of the pairand a volume represented by the second object of the pair is above apredefined threshold volume difference, and/or (ii) a difference betweenin a height [e.g. a thickness of the object in a z-direction; e.g.distance of the object along the backbone (e.g. determined as an averagez-coordinate of the object)] of the first object of the pair and aheight [e.g. a thickness of the object in a z-direction; e.g. distanceof the object along the backbone (e.g. determined as an averagez-coordinate of the object)] of the second object of the pair is above apredefined threshold height difference; and following application of thepair comparison filter, selecting, by the processor, each seed rib boneartificial object from the set of prospective seed rib bone artificialobjects.

In certain embodiments, the method comprises verifying whether thedistance (e.g. a Euclidean distance; e.g. a distance along a particularcoordinate (e.g. a z coordinate)) between consecutive seed rib boneartificial objects (e.g. pairs of seed rib bone artificial objects, thefirst and second seed rib bone artificial objects of the pair havingconsecutive rib number indices; e.g. pairs of seed rib bone artificialobjects, the first and second seed rib bone artificial objects of thepair corresponding to nearest neighbors in terms of associatesz-coordinates) is consistent (e.g. substantially the same from pair topair, e.g., within a given tolerance).

In certain embodiments, the method comprises receiving, by theprocessor, a first image of the subject and a second image of thesubject; identifying, by the processor, a plurality of cross-image pairsof artificial objects, each cross-image pair of artificial objectscomprising a first artificial object of a first set of artificialobjects of the first image and a corresponding second artificial objectof a second set of artificial objects of the second image; for eachcross-image pair of artificial objects, determining, by the processor, adifference between a volume of the first artificial object and a volumeof the second artificial object; and eliminating, by the processor, fromthe respective set of artificial objects of each respective image,artificial objects of cross-image pairs for which the determined volumedifference is above a predefined threshold difference.

In certain embodiments, the method comprises: determining, by theprocessor, based on the set of artificial objects within the image, animage registration transformation [e.g., a linear transformation (e.g.,a rigid transformation (e.g., a translation; e.g., a rotation); e.g., anaffine transformation; e.g., any combination of one or more rigidtransformations and/or affine transformations); e.g., a nonlineartransformation (e.g., a regularized or smoothed extrapolation ofdistortion field; e.g., a thin-plate spline; e.g., a transformationcomprising an extrapolation operation and/or a smoothing operation);e.g., a combination of one or more linear transformations and one ormore nonlinear transformations]; and applying, by the processor, theimage registration transformation to a region of the 3-D image, therebyregistering the 3-D image (e.g., correcting distortion in the imageand/or co-registering the 3-D image with one or more different 3-Dimages of the subject).

In certain embodiments, the image registration transformation yieldssymmetrization of the 3-D image, thereby correcting distortions in theimage.

In certain embodiments, the received image corresponds to a first image,and the image registration transformation aligns the first image with asecond image of the subject, thereby co-registering the first image withthe second image.

In certain embodiments determining the image registration transformcomprises: determining, by the processor, from the set of artificialobjects within the image, a set of artificial landmarks within theimage, each artificial landmark corresponding to a point determined froma corresponding artificial object; and determining, by the processor,the image registration transformation [e.g., a linear transformation(e.g., a rigid transformation (e.g., a translation; e.g., a rotation);e.g., an affine transformation; e.g., any combination of one or morerigid transformations and/or affine transformations); e.g., a nonlineartransformation (e.g., a regularized or smoothed extrapolation ofdistortion field; e.g., a thin-plate spline; e.g., a transformationcomprising an extrapolation operation and/or a smoothing operation);e.g., a combination of one or more linear transformations and one ormore nonlinear transformations] using the set of artificial landmarkswithin the image and a set of target landmarks, wherein the imageregistration transform is determined to, when applied to pointscorresponding to the artificial landmarks within the image,substantially optimize alignment of the set of artificial landmarks withthe set of target landmarks.

In certain embodiments, the set of target landmarks is symmetric [e.g.,the set of target landmarks comprises a plurality of right side targetlandmarks associated with one or more right ribs of the subject and foreach right side target landmark, a matching left side target landmarkassociated with a matching left rib, wherein for each right side targetlandmark, the position of the right side target landmark maps to theposition of the matching left side target landmark under a mirroroperation applied to the image (e.g., a mirroring about a y-z plane),and for each left side target landmark, the position of the left sidetarget landmark maps to the position of the matching right side targetlandmark under a mirror operation applied to the image (e.g., amirroring about a y-z plane); e.g., the set of target landmarkscomprises a plurality of breastbone target landmarks, wherein a positionof each breastbone target landmark is substantially invariant under amirror operation applied to the image (e.g., a mirroring about a y-zplane)].

In certain embodiments, the method comprises determining, by theprocessor, the set of target landmarks using the set of artificiallandmarks within the image.

In certain embodiments, the set of target landmarks is a set ofpredetermined target landmarks (e.g., determined from one or more 3-Dimages of the subject different from the received image; e.g.,determined from a plurality of 3-D images comprising one or more 3-Dimages of different subjects).

In certain embodiments, the received 3-D image comprises one or moreregions corresponding to graphical representations of soft tissue (e.g.,lungs, heart, etc.), and the method further comprises applying by theprocessor, the image registration transformation to the one or moreregions of the image corresponding to graphical representations of softtissue (e.g., lungs, heart, etc.), thereby registering the soft tissueregions (e.g., correcting distortion in the soft tissue regions and/orco-registering the soft tissue regions with one or more different 3-Dimages of the subject).

In certain embodiments, the received 3-D image of the subject comprisinga graphical representation of a plurality of rib bones and a backbonecorresponds to a first image recorded via a first modality (e.g.,microCT), and, the method comprises: receiving, by the processor, asecond image recorded via a second modality (e.g., different from thefirst modality; e.g., PET; e.g., an optical imaging modality (e.g.,FMT)); determining, by the processor, based on the set of artificialobjects within the image, a first image registration transformation; anddetermining, by the processor, based on the first image registrationtransformation, a second image registration transformation; andapplying, by the processor, the second image registration transformationto a region of the second image.

In certain embodiments, the second image is recorded at substantiallythe same time as the first image, with the subject in a substantiallysimilar pose and/or position (e.g., the subject is in a fixed poseand/or position during recording of the first and second image; e.g.,the first and second images are recorded using a multimodal imagingsystem).

In certain embodiments, the second image registration transformation isthe same as the first image registration transformation.

In certain embodiments, coordinates of a plurality of points of thefirst image are related to coordinates of a plurality of points of thesecond image via a known functional relationship (e.g., based on a knownspatial relationship between the first and second imaging modalities).

In another aspect, the invention is directed to a method forregistration of one or more 3-D images of a subject, the methodcomprising: (a) receiving, by a processor of a computing device, a 3-Dimage of the subject (e.g., a tomographic image, e.g., an image obtainedvia CT, micro-CT, SPECT, x-ray, MR, ultrasound, and/or a combination ofany of these), wherein the 3-D image comprises a graphicalrepresentation of one or more bones of interest (e.g., axial skeletonbones of the subject; e.g., a plurality of rib bones and/or a backbone;e.g., a breastbone); (b) identifying in the graphical representation[e.g., graphically isolating; e.g., automatically (e.g., viasegmentation); e.g., via a user interaction], by the processor, one ormore bones of interest (e.g., a plurality of rib bones; e.g., abreastbone) and a reference object (e.g., a backbone; e.g., a lower endof a breastbone); (c) automatically generating, by the processor, one ormore seed artificial objects, each seed artificial object being within afirst distance interval from the reference object (e.g., (i) for eachpoint of the seed artificial object, a shortest distance from the pointto a surface of the reference object is within the first distanceinterval; e.g., (ii) for each point of the seed artificial object, ashortest distance from the point to a centerline of the reference objectis within the first distance interval) and corresponding to a region ofa bone of interest in the graphical representation; (d) for eachautomatically generated seed artificial object, automaticallygenerating, by the processor, a plurality of associated subsequentartificial objects, thereby creating a set of artificial objects (e.g.,the set comprising the seed and subsequent rib bone artificial objects)within the image; (e) automatically performing, by the processor,registration of one or more images of the subject using the set ofartificial objects within the image (e.g., using a set of artificiallandmarks, each landmark determined based on its correspondingartificial objects within the image, e.g. wherein each artificiallandmark of the set of artificial landmarks is calculated as a center ofmass of each artificial object of the set of artificial objects).

In certain embodiments, the step of automatically generating the one ormore seed artificial objects comprises: determining a distance mapcomprising intensity (e.g., numeric) values at each of a plurality ofpoints in three dimensions, each of the intensity values of the distancemap corresponding to a distance from a given point in 3-D space to thereference object (e.g., (i) a shortest distance from the given point toa surface of the identified reference object; e.g., (ii) a shortestdistance from the given point to a centerline of the identifiedreference object); generating, by the processor, a distance intervalmask from the distance map (e.g., the distance interval mask is a binarymask, wherein points having a distance to the reference object within adistance interval (e.g., the first distance interval) are assigned afirst value (e.g., a numeric 1; e.g., a Boolean true), and other pointsare assigned a second value (e.g., a numeric 0; e.g. a Boolean false));and applying, by the processor, an AND operation (e.g., a point-wise ANDoperation) between the distance interval mask and a mask correspondingto the identified one or more bones of interest, thereby identifying aplurality of regions of the identified bones of interest that are withinthe first distance interval from the reference object.

In certain embodiments, for each automatically generated seed artificialobject, the plurality of associated subsequent artificial objectscomprises an equidistant chain of artificial objects.

In certain embodiments, each artificial object is a predefined thresholddistance from (e.g., distance from the surface of; e.g., distance from acenterline of) an identified additional bone (e.g., a breastbone) withinthe graphical representation.

In certain embodiments, the method comprises identifying, by theprocessor, one or more dangerous regions of the image corresponding toregions in which a distance between a first and second bone of interestin the graphical representation is below a predefined threshold distance(e.g., automatically identifying the one or more dangerous regions ofthe image by applying one or more morphological operations (e.g.,including a morphological closing operation) and one or more logicaloperations to a mask corresponding to the identified bones of interest),and wherein the method comprises ensuring that each automaticallygenerated artificial object is sufficiently far from any identifieddangerous region within the image (e.g., is at least a predefined thirdthreshold distance from the nearest identified dangerous region, e.g.,for each point of each artificial object, a shortest distance from thepoint to the identified dangerous region is at least the third thresholddistance).

In certain embodiments, automatically generating, by the processor, theplurality of subsequent artificial objects associated with a respectiveseed artificial object comprises generating a chain of artificialobjects along a bone of interest by beginning with the seed artificialobject and, in a stepwise fashion, generating new subsequent artificialobjects, each newly generated artificial object proceeding outwards(e.g., a predefined distance from the previously generated rib boneartificial object), away from the identified reference object, along thebone of interest.

In certain embodiments, generating the chain of artificial objectsassociated with the respective seed artificial object comprises, for atleast one newly generated artificial object of the chain of artificialobjects: determining whether the newly generated artificial object iswithin a predetermined threshold distance from an identified additionalbone (e.g., a breastbone) in the graphical representation; andresponsive to determining that the newly generated artificial object iswithin the predetermined threshold distance from the identifiedadditional bone (e.g., the breastbone) of the image, terminatinggeneration of subsequent artificial objects associated with therespective seed artificial object.

In certain embodiments, the method comprises identifying, by theprocessor, one or more dangerous regions of the image corresponding toregions in which a distance between a first and second bone of interestin the graphical representation is below a predefined thresholddistance, and generating the chain of artificial objects associated withthe respective seed artificial object comprises, for at least one newlygenerated artificial object of the chain of artificial objects:determining whether the newly generated artificial object is within apredetermined threshold distance from an identified dangerous region ofthe image; and responsive to determining that the newly generatedartificial object is within the predetermined threshold distance fromthe identified dangerous region of the image, terminating generation ofsubsequent artificial objects associated with the respective seedartificial object.

In certain embodiments, each artificial object of the set of artificialobjects within the image (e.g. each rib bone artificial object and/oreach breastbone artificial object) is confirmed to have at least apredefined threshold volume.

In certain embodiments, automatically generating the one or more seedartificial objects comprises: generating, by the processor, a set ofprospective seed artificial objects corresponding to plurality ofregions of the one or more bones of interest that are within the firstdistance interval from the reference object; applying, by the processor,a volume filter to the set of prospective seed artificial objects,wherein application of the volume filter eliminates, from the set, thoseartificial objects determined, by the processor, to represent a volumebelow a predefined threshold volume; and following application of thevolume filter, selecting, by the processor, the seed artificial objectfrom the set of prospective seed artificial objects.

In certain embodiments, the method comprises: receiving, by theprocessor, a first image of the subject and a second image of thesubject; identifying, by the processor, a plurality of cross-image pairsof artificial objects, each cross-image pair of artificial objectscomprising a first artificial object of a first set of artificialobjects of the first image and a corresponding second artificial objectof a second set of artificial objects of the second image; for eachcross-image pair of artificial objects, determining, by the processor, adifference between a volume of the first artificial object and a volumeof the second artificial object; and eliminating, by the processor, fromthe respective set of artificial objects of each respective image,artificial objects of cross-image pairs for which the determined volumedifference is above a predefined threshold difference.

In certain embodiments, the method comprises: determining, by theprocessor, based on the set of artificial objects within the image, animage registration transformation [e.g. a linear transformation (e.g. arigid transformation (e.g. a translation, e.g. a rotation), an affinetransformation, e.g. any combination of one or more rigidtransformations and/or affine transformations); e.g. a nonlineartransformation (a regularized or smoothed extrapolation of distortionfield, e.g. a thin-plate spline, e.g. a transformation comprising anextrapolation operation and/or a smoothing operation); e.g. acombination of one or more linear transformations and one or morenonlinear transformations]; and applying, by the processor, the imageregistration transformation to a region of the 3-D image, therebyregistering the 3-D image (e.g. correcting distortion in the imageand/or co-registering the 3-D image with one or more different 3-Dimages of the subject).

In certain embodiments, the image registration transformation yieldssymmetrization of the 3-D image, thereby correcting distortions in theimage.

In certain embodiments, the received image corresponds to a first image,and the image registration transformation aligns the first image with asecond image of the subject, thereby co-registering the first image withthe second image.

In certain embodiments, determining the image registrationtransformation comprises: determining, by the processor, from the set ofartificial objects within the image, a set of artificial landmarkswithin the image, each landmark corresponding to a point determined froma corresponding artificial object; and determining, by the processor,the image registration transformation [e.g. a linear transformation(e.g. a rigid transformation (e.g. a translation, e.g. a rotation), anaffine transformation, e.g. any combination of one or more rigidtransformations and/or affine transformations); e.g. a nonlineartransformation (a regularized or smoothed extrapolation of distortionfield, e.g. a thin-plate spline, e.g. a transformation comprising anextrapolation operation and/or a smoothing operation); e.g. acombination of one or more linear transformations and one or morenonlinear transformations] using the set of artificial landmarks withinthe image and a set of target landmarks, wherein the image registrationtransformation is determined to, when applied to points corresponding tothe artificial landmarks within the image, substantially optimizealignment of the set of artificial landmarks with the set of targetlandmarks.

In certain embodiments, the set of target landmarks is symmetric [e.g.the set of target landmarks comprises a plurality of right side targetlandmarks associated with one or more bones of interest on a right sideof the subject and for each right side target landmark, a matching leftside target landmark associated with a matching bone of interest on aleft side of the subject, wherein for each right side target landmark,the position of the right side target landmark maps to the position ofthe matching left side target landmark under a mirror operation appliedto the image (e.g. a mirroring about a y-z plane), and for each leftside target landmark, the position of the left side target landmark mapsto the position of the matching right side target landmark under amirror operation applied to the image (e.g. a mirroring about a y-zplane)].

In certain embodiments, the method comprises determining, by theprocessor, the set of target landmarks using the set of artificiallandmarks within the image.

In certain embodiments, the set of target landmarks is a set ofpredetermined target landmarks (e.g. determined from one or more 3-Dimages of the subject different from the received image; e.g. determinedfrom a plurality of 3-D images comprising one or more 3-D images ofdifferent subjects).

In certain embodiments, the received 3-D image comprises one or moreregions corresponding to graphical representations of soft tissue (e.g.lungs, heart, etc.), and the method further comprises applying by theprocessor, the image registration transformation to the one or moreregions of the image corresponding to graphical representations of softtissue (e.g. lungs, heart, etc.), thereby registering the soft tissueregions (e.g. correcting distortion in the soft tissue regions and/orco-registering the soft tissue regions with one or more different 3-Dimages of the subject).

In certain embodiments, the received 3-D image of the subjectcorresponds to a first image recorded via a first modality (e.g.microCT), and, the method comprises: receiving, by the processor, asecond image recorded via a second modality (e.g. different from thefirst modality; e.g. PET; e.g. an optical imaging modality (e.g. FMT));determining, by the processor, based on the set of artificial objectswithin the image, a first image registration transformation; anddetermining, by the processor, based on the first image registrationtransformation, a second image registration transformation; andapplying, by the processor, the second image registration transformationto a region of the second image.

In certain embodiments, the second image is recorded at substantiallythe same time as the first image, with the subject in a substantiallysimilar pose and/or position (e.g. the subject is in a fixed pose and/orposition during recording of the first and second image; e.g. the firstand second images are recorded using a multimodal imaging system).

In certain embodiments, the second image registration transformation isthe same as the first image registration transformation.

In certain embodiments, coordinates of a plurality of points of thefirst image are related to coordinates of a plurality of points of thesecond image via a known functional relationship (e.g. based on a knownspatial relationship between the first and second imaging modalities).

In certain embodiments, the 3-D image comprises a graphicalrepresentation of a plurality of rib bones and a backbone, theidentified one or more bones of interest comprise a plurality of ribbones, and the identified reference object is a backbone of the subject.

In certain embodiments, the plurality of rib bones includes one or morepairs of opposite rib bones (e.g. each pair of opposite rib bonescomprising a right rib bone and a corresponding left rib bone), each ribbone artificial object of a portion of the plurality of rib boneartificial objects belongs to one of a plurality of pairs of rib boneartificial objects, each pair comprising a first rib bone artificialobject associated with a first rib bone (e.g. a right rib bone) of apair of opposite rib bones and a second rib bone artificial objectassociated with a second rib bone (e.g. a left rib bone) of the pair ofopposite rib bones, and for each pair of rib bone artificial objects,(i) a difference between a volume represented by the first object of thepair and a volume represented by the second object of the pair is belowa predefined threshold volume difference, and/or (ii) a differencebetween in a height [e.g. a thickness of the object in a z-direction;e.g. distance of the object along the backbone (e.g. determined as anaverage z-coordinate of the object)] of the first object of the pair anda height [e.g. a thickness of the object in a z-direction; e.g. distanceof the object along the backbone (e.g. determined as an averagez-coordinate of the object)] of the second object of the pair is below apredefined threshold height difference.

In certain embodiments, the plurality of rib bones in the graphicalrepresentation includes one or more pairs of opposite rib bones (e.g.each pair of opposite rib bones comprising a right rib bone and acorresponding left rib bone), and automatically generating the pluralityof seed rib bone artificial objects comprises: identifying, by theprocessor, a set of prospective seed rib bone artificial objectscorresponding to plurality of regions of the identified rib bones thatare within a distance interval from the backbone; automaticallyidentifying, by the processor, one or more pairs of prospective seed ribbone artificial objects, each pair comprising a first seed rib boneartificial object of a first rib bone of the pair of rib bones andcorresponding second seed rib bone artificial object of the opposite ribbone; applying, by the processor, a pair comparison filter to the set ofprospective seed rib bone artificial objects, wherein application of thepair comparison filter eliminates, from the set, pairs of artificialobjects for which (i) a difference between a volume represented by thefirst object of the pair and a volume represented by the second objectof the pair is above a predefined threshold volume difference, and/or(ii) a difference between in a height [e.g. a thickness of the object ina z-direction; e.g. distance of the object along the backbone (e.g.determined as an average z-coordinate of the object)] of the firstobject of the pair and a height [e.g. a thickness of the object in az-direction; e.g. distance of the object along the backbone (e.g.determined as an average z-coordinate of the object)] of the secondobject of the pair is above a predefined threshold height difference;and following application of the pair comparison filter, selecting, bythe processor, each seed rib bone artificial object from the set ofprospective seed rib bone artificial objects.

In certain embodiments, the 3-D image comprises a graphicalrepresentation of a breastbone of the subject and a lower end of thebreastbone.

In certain embodiments, the one or more bones of interest comprise(s)the breastbone in the graphical representation and the reference objectcomprises a lower end of the breastbone.

In another aspect, the invention is directed to a system forregistration of one or more 3-D images of a subject, the systemcomprising: a processor; and a memory having instructions storedthereon, wherein the instructions, when executed by the processor, causethe processor to: (a) receive a 3-D image of the subject (e.g., atomographic image; e.g., an image obtained via CT, micro-CT, SPECT,x-ray, MR, ultrasound, and/or a combination of any of these), whereinthe 3-D image comprises a graphical representation of rib bones and abackbone; (b) identify in the graphical representation [e.g.,graphically isolating, e.g. automatically (e.g. via segmentation); e.g.via a user interaction] (i) the backbone and (ii) ribcage bonescomprising the rib bones [e.g., by determination of a bone mask followedby separation of identified bone tissue into a first part correspondingto an identified backbone and a second part corresponding to identifiedrib bones [{e.g., by determination of a bone mask followed by separationof identified bone tissue into a first part corresponding to anidentified backbone and a second part corresponding to identified ribbones (e.g., and, optionally, a third part corresponding to anidentified breastbone)} {e.g., by determination of a bone mask followedby separation of identified bone tissue into a first part correspondingto an identified backbone and a second part corresponding to identifiedribcage bones (e.g., comprising rib bones together with a breastbone),optionally followed by separation of the identified ribcage bones intoan identified breastbone and a part corresponding to the remaining bones(i.e., the rib bones)}]; (c) automatically generate a plurality of seedrib bone artificial objects, each seed rib bone artificial object beingwithin a first distance interval from the backbone in the graphicalrepresentation (e.g., (i) for each point of the seed rib bone artificialobject, a shortest distance from the point to a surface of theidentified backbone is within the first distance interval; e.g., (ii)for each point of the seed rib bone artificial object, a shortestdistance from the point to a centerline of the identified backbone iswithin the first distance interval) and corresponding to a region of arib bone in the graphical representation (e.g., wherein each seed ribbone artificial object is assigned a left-right index and a rib numberindex); (d) for each automatically generated seed rib bone artificialobject, automatically generate a plurality of associated subsequent ribbone artificial objects, thereby creating a set of artificial objects(e.g., the set comprising the seed and subsequent rib bone artificialobjects) within image; (e) automatically perform registration of one ormore images of the subject using the set of artificial objects withinthe image (e.g., using a set of artificial landmarks, each landmarkdetermined based on its corresponding artificial object within theimage; e.g., wherein each artificial landmark of the set of artificiallandmarks is calculated as a center of mass of each artificial object ofthe set of artificial objects).

In certain embodiments, identifying the back bone and rib bonescomprises determining a set of one or more bone masks to identifyspecific bones in the graphical representation. For example, determiningthe set of bone masks may be conducted in multiple steps—e.g., a firststep to identify all of the bones in the graphical representation, asecond step to identify the backbone, and a third step to identify(undifferentiated) ribcage bones comprising the rib bones. In certainembodiments, the identified ribcage bones include rib bones togetherwith a breastbone. In certain embodiments, a fourth step is performedthat separates the identified ribcage bones into (i) an identifiedbreastbone and (ii) remaining bones corresponding to identified ribbones. In certain embodiments, it is not necessary to separatelyidentify the breastbone and rib bones, and the identified ribcage bonescorrespond to an undifferentiated collection of rib bones together withthe breast bone. In certain embodiments, the breast bone and rib bonesare separately identified, and operations described herein as beingperformed on identified ribcage bones and/or a mask corresponding toidentified ribcage bones are performed on identified rib bones and/or amask corresponding to identified rib bones. In certain embodiments, itis not necessary to separately identify individual rib bones (thoughthis may be performed in other embodiments), but rather, anundifferentiated collection of rib bones is identified.

In certain embodiments, the instructions cause the processor toautomatically generate the plurality of seed rib bone artificial objectsby: determining a backbone distance map comprising intensity (e.g.numeric, e.g. non-negative numeric) values at each of a plurality ofpoints in three dimensions, each of the intensity values of the backbonedistance map corresponding to a distance from the given point in 3-Dspace to the identified backbone in the graphical representation (e.g.,(i) a shortest distance from the given point to the identified backbone;e.g., (ii) a shortest distance from the given point to a centerline ofthe identified backbone); generating a backbone distance interval maskfrom the backbone distance map, the backbone distance interval maskcorresponding to regions of the image that are within the first distanceinterval from the identified backbone [e.g., the distance interval maskis a binary mask, wherein points having a distance to the identifiedbackbone within a distance interval (e.g., the first distance interval)are assigned a first value (e.g., a numeric 1; e.g. a Boolean true), andother points are assigned a second value (e.g., a numeric 0; e.g., aBoolean false)]; and applying an AND operation (e.g., a point-wise ANDoperation) between the backbone distance interval mask and a maskcorresponding to the identified ribcage bones (e.g., the identified ribbones), thereby identifying a plurality of regions of the identifiedribcage bones that are within the first distance interval from theidentified backbone.

In certain embodiments, at step (c), the instructions cause theprocessor to, for each of the plurality of seed rib bone artificialobjects, associate with the seed rib bone artificial object (i) aleft-right index (e.g., based on an x-coordinate determined from one ormore points of the seed rib bone artificial object) and (ii) a ribnumber index (e.g., based on a z-coordinate determined from one or morepoints of the seed rib bone artificial object).

In certain embodiments, associating the seed rib bone artificial objectwith a left-right index comprises, as a preliminary step of applying apreliminary rotation and translation transformation; and, subsequently,determining the left-right index (e.g. based on an x-coordinatedetermined from one or more points of the seed rib bone artificialobject). In certain embodiments, the preliminary rotation andtranslation transformation is determined using principal axes of theidentified backbone and/or principal axes of an identified breastbone inthe graphical representation.

In certain embodiments, for each automatically generated seed rib boneartificial object, the plurality of associated subsequent rib boneartificial objects comprises an equidistant chain of rib bone artificialobjects.

In certain embodiments, the 3-D image comprises a graphicalrepresentation of rib bones, a backbone, and a breastbone, and each ribbone artificial object is at least a predefined first threshold distancefrom an identified breastbone in the graphical representation (e.g., foreach point of each rib bone artificial object, a shortest distance fromthe point to the identified breastbone is at least the first thresholddistance; e.g., for each point of each rib bone artificial object, ashortest distance from the point to a centerline of the identifiedbreastbone is at least the first threshold distance).

In certain embodiments, the instructions cause the processor to:automatically identify one or more dangerous regions of the imagecorresponding to regions in which a distance between a first and secondrib bone in the graphical representation is below a predefined secondthreshold distance [e.g., automatically identifying the one or moredangerous regions of the image by applying one or more morphologicaloperations (e.g. including a morphological closing operation) and one ormore logical operations to a mask corresponding to the identifiedribcage bones]; and generate each rib bone artificial object such thatit is sufficiently far from any identified dangerous region within theimage (e.g., is at least a predefined third threshold distance from thenearest identified dangerous region; e.g., for each point of each ribbone artificial object, a shortest distance from the point to theidentified dangerous region is at least the third threshold distance).

In certain embodiments, the instructions cause the processor toautomatically generate the plurality of subsequent rib bone artificialobjects associated with a respective seed rib bone artificial object bygenerating a chain of rib bone artificial objects along a rib bone bybeginning with the respective seed rib bone artificial object and, in astepwise fashion, generating new subsequent rib bone artificial objects,each newly generated rib bone artificial object proceeding outwards(e.g., a predefined distance from the previously generated rib boneartificial object), away from the identified backbone, along the ribbone.

In certain embodiments, the 3-D image comprises a graphicalrepresentation of rib bones, a backbone, and a breastbone, and theinstructions cause the processor to generate the chain of artificial ribbone objects associated with the respective seed rib bone artificialobject by, for at least one newly generated rib bone artificial objectof the chain of rib bone artificial objects: determining whether thenewly generated rib bone artificial object is within a predeterminedfirst threshold distance from an identified breastbone in the graphicalrepresentation; and responsive to determining that the newly generatedrib bone artificial object is within the predetermined first thresholddistance from the identified breastbone, terminating generation ofsubsequent artificial objects associated with the respective seed ribbone artificial object.

In certain embodiments, the instructions cause the processor to:identify one or more dangerous regions of the image corresponding toregions in which a distance between a first and second rib bone in thegraphical representation is below a predefined second thresholddistance; and generate the chain of rib bone artificial objectsassociated with the respective seed rib bone artificial object, by, forat least one newly generated rib bone artificial object of the chain ofrib bone artificial objects: determining whether the newly generated ribbone artificial object is within a predetermined third thresholddistance from an identified dangerous region of the image; andresponsive to determining that the newly generated rib bone artificialobject is within the third predetermined threshold distance from theidentified dangerous region of the image, terminating generation ofsubsequent rib bone artificial objects associated with the respectiveseed rib bone artificial object.

In certain embodiments, the instructions cause the processor toautomatically identify the one or more dangerous regions by applying oneor more morphological closing operations to a mask corresponding toidentified ribcage bones.

In certain embodiments, the 3-D image comprises a graphicalrepresentation of at least a portion of a breastbone of the subject, theportion comprising a lower (e.g. tail-sided) end of the breastbone, andwherein the instructions cause the processor to automatically generate aseries of artificial objects along the breastbone by: (f) identifying inthe graphical representation [e.g., graphically isolating, e.g.,automatically (e.g. via segmentation), e.g., via a user interaction[thebreastbone [{e.g., by determination of a bone mask followed byseparation of identified bone tissue into the backbone, a breastbone,and remaining bones (i.e., the rib bones)} {e.g., by determination of abone mask followed by separation of identified bone tissue into a firstpart corresponding to an identified backbone and a second partcorresponding to identified ribcage bones (e.g., comprising rib bonestogether with a breastbone), followed by separation of the identifiedribcage bones into an identified breastbone and a part corresponding tothe remaining bones (i.e., the rib bones)}] and the lower end of thebreastbone (e.g., by determining a minimal z-coordinate of a breastbonemask corresponding to the identified breastbone); (g) automaticallygenerating a seed breastbone artificial object, the seed breastboneartificial object corresponding to a region of the identified breastbonethat is within a second distance interval from the lower end of thebreastbone [e.g., (i) for each point of the seed breastbone artificialobject, a shortest distance from the point to a surface of theidentified lower end of the breastbone is within the second distanceinterval; e.g., (ii) for each point of the seed breastbone artificialobject, a distance (e.g., a Euclidean distance; e.g., a distance along aparticular coordinate (e.g., a z-coordinate)) from the point to a pointcorresponding to the identified lower end of the breastbone is withinthe second distance interval]; and (h) beginning with the seedbreastbone artificial object, automatically generating a plurality ofassociated breastbone artificial objects along the identifiedbreastbone, the plurality of breastbone artificial objects includedwithin the set of artificial objects within the image (e.g., therebycreating a set of artificial objects further comprising the generatedbreastbone artificial objects).

In certain embodiments, the instructions cause the processor toautomatically generate the seed breastbone artificial object by:determining a breastbone lower end distance map comprising intensity(e.g., numeric) values at each of a plurality of points in threedimensions, each of the intensity values of the backbone distance mapcorresponding to a distance from a given point in 3-D space to theidentified lower end of the breastbone in the graphical representation[e.g., (i) a shortest distance from the given point to a surface of theidentified lower end of the breastbone; e.g., (ii) a distance (e.g., aEuclidean distance, e.g., a distance along a particular coordinate(e.g., a z-coordinate)) from the given point to a point correspondingthe identified lower end of the breastbone]; generating a breastbonelower end distance interval mask from the breastbone lower end distancemap, the breastbone lower end distance interval mask corresponding toregions of the image that are within the second distance interval fromthe lower end of the breastbone [e.g., the breastbone lower end distanceinterval mask is a binary mask, wherein points having a distance to thelower end of the breastbone within a predefined distance interval (e.g.,the second distance interval) are assigned a first value (e.g., anumeric 1; e.g., a Boolean true), and other points are assigned a secondvalue (e.g., a numeric 0; e.g. a Boolean false)]; and applying, an ANDoperation (e.g., a point-wise AND operation) between the breastbonelower end distance interval mask and a mask corresponding to theidentified breastbone, thereby identifying a region of the identifiedbreastbone that is within the second distance interval from the lowerend of the breastbone.

In certain embodiments, the plurality of breastbone artificial comprisesan equidistant chain of breastbone artificial objects along thebreastbone.

In certain embodiments, the instructions cause the processor toautomatically generate the plurality of breastbone artificial objects bygenerating a chain of breastbone artificial object by beginning with theseed breastbone artificial object and, in a stepwise fashion, generatingnew subsequent breastbone artificial objects, each newly generatedbreastbone artificial object proceeding upwards (e.g., a predefineddistance from the previously generated breastbone artificial object),away from the identified lower end of the breastbone.

In certain embodiments, the portion of the breastbone in the graphicalrepresentation comprises an upper end of the breastbone, and theinstructions cause the processor to: identify in the graphicalrepresentation, by the processor, the upper end of the breastbone; andgenerate the chain of breastbone artificial objects by, for at least onenewly generated breastbone artificial object: determining whether thenewly generated breastbone artificial object reaches or is within apredetermined threshold distance from the identified upper end of thebreastbone; and responsive to determining that the newly generatedbreastbone artificial object reaches or is within the predeterminedthreshold distance from the identified upper end of the breastbone,terminating generation of subsequent breastbone artificial objects.

In certain embodiments, each automatically generated artificial objectof the set of artificial objects within the image (e.g., each rib boneartificial object and/or each breastbone artificial object) is confirmedto have at least a predefined threshold volume.

In certain embodiments, the instructions cause the processor toautomatically generate the plurality of seed rib bone artificial objectsby: generating a set of prospective seed rib bone artificial objectscorresponding to plurality of regions of the rib bones in the graphicalrepresentation that are within a distance interval from the backbone inthe graphical representation; applying a volume filter to the set ofprospective seed rib bone artificial objects, wherein application of thevolume filter eliminates, from the set, those artificial objectsdetermined, by the processor, to represent a volume below a predefinedthreshold volume; and following application of the volume filter,selecting each seed rib bone artificial object from the set ofprospective seed rib bone artificial objects.

In certain embodiments, the rib bones in the graphical representationinclude a plurality [e.g., corresponding to a number of rib bones on agiven side of the subject (e.g., a known number of rib bones)] of pairsof opposite rib bones (e.g., each pair of opposite rib bones comprisinga right rib bone and a corresponding left rib bone), each rib boneartificial object of a portion of the plurality of rib bone artificialobjects belongs to one of a plurality of pairs of rib bone artificialobjects, each pair comprising a first rib bone artificial objectassociated (e.g., based on an associated left-right index of rib boneartificial object) with a first rib bone (e.g., a right rib bone) of apair of opposite rib bones and a second rib bone artificial objectassociated (e.g. based on an associated left-right index of rib boneartificial object) with a second rib bone (e.g., a left rib bone) of thepair of opposite rib bones, and for each pair of rib bone artificialobjects, (i) a difference between a volume represented by the firstobject of the pair and a volume represented by the second object of thepair is below a predefined threshold volume difference, and/or (ii) adifference between a height [e.g., a thickness of the object in az-direction; e.g. distance of the object along the backbone (e.g.,determined as an average z-coordinate of the object)] of the firstobject of the pair and a height [e.g., a thickness of the object in az-direction; e.g., distance of the object along the backbone (e.g.,determined as an average z-coordinate of the object)] of the secondobject of the pair is below a predefined threshold height difference.

In certain embodiments, the rib bones in the graphical representationinclude a plurality of pairs of opposite rib bones (e.g., each pair ofopposite rib bones comprising a right rib bone and a corresponding leftrib bone), and the instructions cause the processor to automaticallygenerate the plurality of seed rib bone artificial object by: generatinga set of prospective seed rib bone artificial objects corresponding toplurality of regions of the rib bones in the graphical representationthat are within the first distance interval from the backbone;automatically identifying one or more pairs of prospective seed rib boneartificial objects, each pair comprising a first seed rib boneartificial object of a first rib bone of the pair of rib bones andcorresponding second seed rib bone artificial object of the opposite ribbone (e.g., based on z-coordinates of each seed rib bone artificialobject; e.g., based on an associated rib number index of each seed ribbone artificial object; e.g., based on an associated left-right index ofeach seed rib bone artificial object); applying a pair comparison filterto the set of prospective seed rib bone artificial objects, whereinapplication of the pair comparison filter eliminates, from the set,pairs of artificial objects for which (i) a difference between a volumerepresented by the first object of the pair and a volume represented bythe second object of the pair is above a predefined threshold volumedifference, and/or (ii) a difference between in a height of the firstobject of the pair and a height of the second object of the pair isabove a predefined threshold height difference; and followingapplication of the pair comparison filter, selecting each seed rib boneartificial object from the set of prospective seed rib bone artificialobjects.

In certain embodiments, the instructions cause the processor to verifywhether the distance [e.g., a Euclidean distance; e.g., a distance alonga particular coordinate (e.g., a z-coordinate)] between consecutive seedrib bone artificial objects (e.g., pairs of seed rib bone artificialobjects, the first and second seed rib bone artificial objects of thepair having consecutive rib number indices; e.g., pairs of seed rib boneartificial objects, the first and second seed rib bone artificialobjects of the pair corresponding to nearest neighbors in terms ofassociates z-coordinates) is consistent (e.g., substantially the samefrom pair to pair; e.g., within a given tolerance).

In certain embodiments, the instructions cause the processor to: receivea first image of the subject and a second image of the subject; identifya plurality of cross-image pairs of artificial objects, each cross-imagepair of artificial objects comprising a first artificial object of afirst set of artificial objects of the first image and a correspondingsecond artificial object of a second set of artificial objects of thesecond image; for each cross-image pair of artificial objects, determinea difference between a volume of the first artificial object and avolume of the second artificial object; and eliminate, from therespective set of artificial objects of each respective image,artificial objects of cross-image pairs for which the determined volumedifference is above a predefined threshold difference.

In certain embodiments, the instructions cause the processor to:determine, based on the set of artificial objects within the image, animage registration transformation [e.g., a linear transformation (e.g.,a rigid transformation (e.g., a translation; e.g., a rotation); e.g., anaffine transformation; e.g., any combination of one or more rigidtransformations and/or affine transformations); e.g., a nonlineartransform (e.g., a regularized or smoothed extrapolation of distortionfield; e.g., a thin-plate spline transform; e.g., a transformationcomprising an extrapolation operation and/or a smoothing operation);e.g., a combination of one or more linear transformations and one ormore nonlinear transformations]; and apply the image registrationtransformation to a region of the 3-D image, thereby registering the 3-Dimage (e.g., correcting distortion in the image and/or co-registeringthe 3-D image with one or more different 3-D images of the subject).

In certain embodiments, the image registration transformation yieldssymmetrization of the 3-D image, thereby correcting distortions in theimage.

In certain embodiments, the received image corresponds to a first image,and the image registration transformation aligns the first image with asecond image of the subject, thereby co-registering the first image withthe second image.

In certain embodiments, the instructions cause the processor todetermine the image registration transformation by: determining, fromthe set of artificial objects within the image, a set of artificiallandmarks within the image, each artificial landmark corresponding to apoint determined from a corresponding artificial object; and determiningthe image registration transformation [e.g. a linear transformation(e.g. a rigid transformation (e.g. a translation, e.g. a rotation), anaffine transformation, e.g. any combination of one or more rigidtransformations and/or affine transformations); e.g. a nonlineartransformation (e.g. a regularized or smoothed extrapolation ofdistortion field, e.g. a thin-plate spline, e.g. a transformationcomprising an extrapolation operation and/or a smoothing operation);e.g. a combination of one or more linear transformations and one or morenonlinear transformations] using the set of artificial landmarks withinthe image and a set of target landmarks, wherein the image registrationtransformation is determined to, when applied to points corresponding tothe artificial landmarks within the image, substantially optimizealignment of the set of artificial landmarks with the set of targetlandmarks.

In certain embodiments, the set of target landmarks is symmetric, [e.g.,the set of target landmarks comprises a plurality of right side targetlandmarks associated with one or more right ribs of the subject and foreach right side target landmark, a matching left side target landmarkassociated with a matching left rib, wherein for each right side targetlandmark, the position of the right side target landmark maps to theposition of the matching left side target landmark under a mirroroperation applied to the image (e.g., a mirroring about a y-z plane),and for each left side target landmark, the position of the left sidetarget landmark maps to the position of the matching right side targetlandmark under a mirror operation applied to the image (e.g., amirroring about a y-z plane); e.g., the set of target landmarkscomprises a plurality of breastbone target landmarks, wherein a positionof each breastbone target landmark is substantially invariant under amirror operation applied to the image (e.g., a mirroring about a y-zplane)].

In certain embodiments, the instructions cause the processor todetermine the set of target landmarks using the set of artificiallandmarks within the image.

In certain embodiments, the set of target landmarks is a set ofpredetermined target landmarks (e.g., determined from one or more 3-Dimages of the subject different from the received image; e.g.,determined from a plurality of 3-D images comprising one or more 3-Dimages of different subjects).

In certain embodiments, the received 3-D image comprises one or moreregions corresponding to graphical representations of soft tissue (e.g.,lungs, heart, etc.), and the instructions cause the processor to applythe image registration transformation to the one or more regions of theimage corresponding to graphical representations of soft tissue (e.g.,lungs, heart, etc.), thereby registering the soft tissue regions (e.g.,correcting distortion in the soft tissue regions and/or co-registeringthe soft tissue regions with one or more different 3-D images of thesubject).

In certain embodiments, the received 3-D image of the subject comprisinga graphical representation of rib bones and a backbone corresponds to afirst image recorded via a first modality (e.g., microCT), and, theinstructions cause the processor to: receive a second image recorded viaa second modality (e.g., different from the first modality; e.g., PET;e.g., an optical imaging modality (e.g., FMT)); determine, based on theset of artificial objects within the image, a first image registrationtransformation; and determine, based on the first image registrationtransformation, a second image registration transform; and apply, thesecond image registration transformation to a region of the secondimage.

In certain embodiments, the second image is recorded at substantiallythe same time as the first image, with the subject in a substantiallysimilar pose and/or position (e.g., the subject is in a fixed poseand/or position during recording of the first and second image; e.g.,the first and second images are recorded using a multimodal imagingsystem).

In certain embodiments, the second image registration transformation isthe same as the first image registration transformation.

In certain embodiments, coordinates of a plurality of points of thefirst image are related to coordinates of a plurality of points of thesecond image via a known functional relationship (e.g., based on a knownspatial relationship between the first and second imaging modalities).

In another aspect, the invention is directed to a system forregistration of one or more 3-D images of a subject, the systemcomprising: a processor; and a memory with instructions stored thereon,wherein the instructions, when executed by the processor, cause theprocessor to: (a) receive a 3-D image of the subject (e.g., atomographic image; e.g., an image obtained via CT, micro-CT, SPECT,x-ray, MR, ultrasound, and/or a combination of any of these), whereinthe 3-D image comprises a graphical representation of one or more bonesof interest (e.g., axial skeleton bones of the subject; e.g., rib bonesand/or a backbone, e.g. a breastbone); (b) identify in the graphicalrepresentation [e.g., graphically isolating; e.g. automatically (e.g.via segmentation); e.g. via a user interaction] one or more bones ofinterest (e.g., ribcage bones (e.g. rib bones together with abreastbone); e.g., rib bones; e.g., a breastbone) and a reference object(e.g., a backbone; e.g., a lower end of a breastbone); (c) automaticallygenerate one or more seed artificial objects, each seed artificialobject being within a first distance interval from the reference object(e.g., (i) for each point of the seed artificial object, a shortestdistance from the point to a surface of the reference object is withinthe first distance interval; e.g., (ii) for each point of the seedartificial object, a shortest distance from the point to a centerline ofthe reference object is within the first distance interval) andcorresponding to a region of a bone of interest in the graphicalrepresentation; (d) for each automatically generated seed artificialobject, automatically generate a plurality of associated subsequentartificial objects, thereby creating a set of artificial objects (e.g.,the set comprising the seed and subsequent rib bone artificial objects)within the image; (e) automatically perform registration of one or moreimages of the subject using the set of artificial objects within theimage [e.g., using a set of artificial landmarks, each landmarkdetermined based on its corresponding artificial objects within theimage (e.g., wherein each artificial landmark of the set of artificiallandmarks is calculated as a center of mass of each artificial object ofthe set of artificial objects)].

In certain embodiments, the instructions cause the processor toautomatically generating the one or more seed artificial objects by:determining a distance map comprising intensity (e.g., numeric; e.g.,non-negative numeric) values at each of a plurality of points in threedimensions, each of the intensity values of the distance mapcorresponding to a distance from a given point in 3-D space to thereference object (e.g., (i) a shortest distance from the given point toa surface of the identified reference object; e.g., (ii) a shortestdistance from the given point to a centerline of the identifiedreference object); generating a distance interval mask from the distancemap [e.g., the distance interval mask is a binary mask, wherein pointshaving a distance to the reference object within a distance interval(e.g., the first distance interval) are assigned a first value (e.g., anumeric 1; e.g., a Boolean true), and other points are assigned a secondvalue (e.g., a numeric 0; e.g., a Boolean false)]; and applying an ANDoperation (e.g., a point-wise AND operation) between the distanceinterval mask and a mask corresponding to the identified one or morebones of interest, thereby identifying a plurality of regions of theidentified bones of interest that are within the first distance intervalfrom the reference object.

In certain embodiments, for each automatically generated seed artificialobject, the plurality of associated subsequent artificial objectscomprises an equidistant chain of artificial objects.

In certain embodiments, each artificial object is a predefined thresholddistance from (e.g., distance from the surface of; e.g., distance from acenterline of) an identified additional bone (e.g., a breastbone) withinthe graphical representation.

In certain embodiments, the instructions cause the processor to:identify one or more dangerous regions of the image corresponding toregions in which a distance between a first and second bone of interestin the graphical representation is below a predefined threshold distance(e.g., automatically identifying the one or more dangerous regions ofthe image by applying one or more morphological operations (e.g.,including a morphological closing operation) and one or more logicaloperations to a mask corresponding to the identified bones of interest);and automatically generate each artificial object such that it issufficiently far from any identified dangerous region within the image(e.g., is at least a predefined third threshold distance from thenearest identified dangerous region; e.g., for each point of eachartificial object, a shortest distance from the point to the identifieddangerous region is at least the third threshold distance).

In certain embodiments, the instructions cause the processor toautomatically generate the plurality of subsequent artificial objectsassociated with a respective seed artificial object by generating achain of artificial objects along a bone of interest by beginning withthe seed artificial object and, in a stepwise fashion, generating newsubsequent artificial objects, each newly generated artificial objectproceeding outwards (e.g., a predefined distance from the previouslygenerated rib bone artificial object), away from the identifiedreference object, along the bone of interest.

In certain embodiments, the instructions cause the processor to generatethe chain of artificial objects associated with the respective seedartificial object by, for at least one newly generated artificial objectof the chain of artificial objects: determining whether the newlygenerated artificial object is within a predetermined threshold distancefrom an identified additional bone (e.g., a breastbone) in the graphicalrepresentation; and responsive to determining that the newly generatedartificial object is within the predetermined threshold distance fromthe identified additional bone (e.g., the breastbone) of the image,terminating generation of subsequent artificial objects associated withthe respective seed artificial object.

In certain embodiments, the instructions cause the processor to:identify one or more dangerous regions of the image corresponding toregions in which a distance between a first and second bone of interestin the graphical representation is below a predefined thresholddistance, and generate the chain of artificial objects associated withthe respective seed artificial object by, for at least one newlygenerated artificial object of the chain of artificial objects:determining whether the newly generated artificial object is within apredetermined threshold distance from an identified dangerous region ofthe image; and responsive to determining that the newly generatedartificial object is within the predetermined threshold distance fromthe identified dangerous region of the image, terminating generation ofsubsequent artificial objects associated with the respective seedartificial object.

In certain embodiments, each artificial object of the set of artificialobjects within the image (e.g., each rib bone artificial object and/oreach breastbone artificial object) is confirmed to have at least apredefined threshold volume.

In certain embodiments, the instructions cause the processor toautomatically generate the one or more seed artificial objects by:generating a set of prospective seed artificial objects corresponding toplurality of regions of the one or more bones of interest that arewithin the first distance interval from the reference object; applying avolume filter to the set of prospective seed artificial objects, whereinapplication of the volume filter eliminates, from the set, thoseartificial objects determined to represent a volume below a predefinedthreshold volume; and following application of the volume filter,selecting the seed artificial objects from the set of prospective seedartificial objects.

In certain embodiments, the instructions cause the processor: to receivea first image of the subject and a second image of the subject; identifya plurality of cross-image pairs of artificial objects, each cross-imagepair of artificial objects comprising a first artificial object of afirst set of artificial objects of the first image and a correspondingsecond artificial object of a second set of artificial objects of thesecond image; for each cross-image pair of artificial objects, determinea difference between a volume of the first artificial object and avolume of the second artificial object; and eliminate, from therespective set of artificial objects of each respective image,artificial objects of cross-image pairs for which the determined volumedifference is above a predefined threshold difference.

In certain embodiments, the instructions cause the processor to:determine, based on the set of artificial objects within the image, animage registration transformation [e.g., a linear transformation (e.g.,a rigid transformation (e.g., a translation; e.g., a rotation); e.g., anaffine transformation; e.g. any combination of one or more rigidtransformations and/or affine transformations); e.g., a nonlineartransformation (e.g., a regularized or smoothed extrapolation ofdistortion field; e.g., a thin-plate spline; e.g., a transformationcomprising an extrapolation operation and/or a smoothing operation);e.g., a combination of one or more linear transformations and one ormore nonlinear transformations]; and apply the image registrationtransformation to a region of the 3-D image, thereby registering the 3-Dimage (e.g., correcting distortion in the image and/or co-registeringthe 3-D image with one or more different 3-D images of the subject).

In certain embodiments, the image registration transformation yieldssymmetrization of the 3-D image, thereby correcting distortions in theimage.

In certain embodiments, the received image corresponds to a first image,and the image registration transformation aligns the first image with asecond image of the subject, thereby co-registering the first image withthe second image.

In certain embodiments, the instructions cause the processor todetermine the image registration transformation by: determining, fromthe set of artificial objects within the image, a set of artificiallandmarks within the image, each landmark corresponding to a pointdetermined from a corresponding artificial object; and determining theimage registration transformation [e.g., a linear transformation (e.g.,a rigid transformation (e.g., a translation; e.g., a rotation); e.g., anaffine transformation; e.g., any combination of one or more rigidtransformations and/or affine transformations); e.g., a nonlineartransformation (e.g., a regularized or smoothed extrapolation ofdistortion field; e.g., a thin-plate spline; e.g., a transformationcomprising an extrapolation operation and/or a smoothing operation);e.g., a combination of one or more linear transformations and one ormore nonlinear transformations] using the set of artificial landmarkswithin the image and a set of target landmarks, wherein the imageregistration transformation is determined to, when applied to pointscorresponding to the artificial landmarks within the image,substantially optimize alignment of the set of artificial landmarks withthe set of target landmarks.

In certain embodiments, the set of target landmarks is symmetric [e.g.,the set of target landmarks comprises a plurality of right side targetlandmarks associated with one or more bones of interest on a right sideof the subject and for each right side target landmark, a matching leftside target landmark associated with a matching bone of interest on aleft side of the subject, wherein for each right side target landmark,the position of the right side target landmark maps to the position ofthe matching left side target landmark under a mirror operation appliedto the image (e.g., a mirroring about a y-z plane), and for each leftside target landmark, the position of the left side target landmark mapsto the position of the matching right side target landmark under amirror operation applied to the image (e.g., a mirroring about a y-zplane)].

In certain embodiments, the instructions cause the processor todetermine the set of target landmarks using the set of artificiallandmarks within the image.

In certain embodiments, the set of target landmarks is a set ofpredetermined target landmarks (e.g., determined from one or more 3-Dimages of the subject different from the received image; e.g.,determined from a plurality of 3-D images comprising one or more 3-Dimages of different subjects).

In certain embodiments, the received 3-D image comprises one or moreregions corresponding to graphical representations of soft tissue (e.g.,lungs, heart, etc.), and the instructions cause the processor to applythe image registration transformation to the one or more regions of theimage corresponding to graphical representations of soft tissue (e.g.,lungs, heart, etc.), thereby registering the soft tissue regions (e.g.,correcting distortion in the soft tissue regions and/or co-registeringthe soft tissue regions with one or more different 3-D images of thesubject).

In certain embodiments, the received 3-D image of the subjectcorresponds to a first image recorded via a first modality (e.g.microCT), and, wherein the instructions cause the processor to: receivea second image recorded via a second modality (e.g., different from thefirst modality; e.g., PET; e.g., an optical imaging modality (e.g.,FMT)); determine, based on the set of artificial objects within theimage, a first image registration transformation; and determine, basedon the first image registration transform, a second image registrationtransformation; and applying the second image registrationtransformation to a region of the second image.

In certain embodiments, the second image is recorded at substantiallythe same time as the first image, with the subject in a substantiallysimilar pose and/or position (e.g., the subject is in a fixed poseand/or position during recording of the first and second image; e.g.,the first and second images are recorded using a multimodal imagingsystem).

In certain embodiments, the second image registration transformation isthe same as the first image registration transformation.

In certain embodiments, coordinates of a plurality of points of thefirst image are related to coordinates of a plurality of points of thesecond image via a known functional relationship (e.g., based on a knownspatial relationship between the first and second imaging modalities).

In certain embodiments, the 3-D image comprises a graphicalrepresentation of rib bones and a backbone, the identified one or morebones of interest comprise rib bones, and the identified referenceobject is a backbone of the subject.

In certain embodiments, the graphical representation of rib bonesincludes a plurality of pairs of opposite rib bones (e.g., each pair ofopposite rib bones comprising a right rib bone and a corresponding leftrib bone), each rib bone artificial object of a portion of the pluralityof rib bone artificial objects belongs to one of a plurality of pairs ofrib bone artificial objects, each pair comprising a first rib boneartificial object associated with a first rib bone (e.g., a right ribbone) of a pair of opposite rib bones and a second rib bone artificialobject associated with a second rib bone (e.g., a left rib bone) of thepair of opposite rib bones, and for each pair of rib bone artificialobjects, (i) a difference between a volume represented by the firstobject of the pair and a volume represented by the second object of thepair is below a predefined threshold volume difference, and/or (ii) adifference between in a height of the first object of the pair and aheight of the second object of the pair is below a predefined thresholdheight difference.

In certain embodiments, the rib bones in the graphical representationincludes one or more pairs of opposite rib bones (e.g., each pair ofopposite rib bones comprising a right rib bone and a corresponding leftrib bone), and wherein the instruction cause the processor toautomatically generate the plurality of seed rib bone artificial objectsby: identifying a set of prospective seed rib bone artificial objectscorresponding to plurality of regions of the rib bones that are within adistance interval from the backbone; automatically identifying one ormore pairs of prospective seed rib bone artificial objects, each paircomprising a first seed rib bone artificial object of a first rib boneof the pair of rib bones and corresponding second seed rib boneartificial object of the opposite rib bone; applying a pair comparisonfilter to the set of prospective seed rib bone artificial objects,wherein application of the pair comparison filter eliminates, from theset, pairs of artificial objects for which (i) a difference between avolume represented by the first object of the pair and a volumerepresented by the second object of the pair is above a predefinedthreshold volume difference, and/or (ii) a difference between in aheight of the first object of the pair and a height of the second objectof the pair is above a predefined threshold height difference; andfollowing application of the pair comparison filter, selecting each seedrib bone artificial object from the set of prospective seed rib boneartificial objects.

In certain embodiments, the 3-D image comprises a graphicalrepresentation of a breastbone of the subject and a lower end of thebreastbone.

In certain embodiments, the one or more bones of interest comprise(s)the breastbone in the graphical representation and the reference objectcomprises a lower end of the breastbone.

BRIEF DESCRIPTION OF THE FIGURES

The foregoing and other objects, aspects, features, and advantages ofthe present disclosure will become more apparent and better understoodby referring to the following description taken in conjunction with theaccompanying drawings, in which:

FIG. 1 is a series of images comprising graphical representations ofribcage bones of a mouse, each image showing a projection of graphicalrepresentations of ribcage bones identified within a different 3-D imageof the mouse recorded at a different time point, according to anillustrative embodiment;

FIG. 2 is a set of two images comprising graphical representations ofribcage bones of a mouse, showing results of distortion correction,according to an illustrative embodiment;

FIG. 3 is a series of images recorded at different times, whereindistortion correction and co-registration has been applied to therecorded images, according to an illustrative embodiment;

FIG. 4 is a set of images that shows the result of application of alinear transformation corresponding to a rotation and the result ofapplication of a non-linear transformation corresponding to a distortionfield correction, according to an illustrative embodiment;

FIG. 5 is a set of images showing a series of automatically generatedartificial landmarks along the breastbone, left rib bones, and right ribbones that are represented in the images, according to an illustrativeembodiment;

FIG. 6 is an intensity map showing a determined distortion correctionfunction representing an x-component of the distortion field, as afunction of image coordinates, according to an illustrative embodiment;

FIG. 7 is a graph showing measured and extrapolated values of adistortion correction function representing an x-component of thedistortion field at locations of a plurality of artificial landmarks,according to an illustrative embodiment;

FIG. 8 is an image showing artificial objects generated along left andright rib bones, according to an illustrative embodiment;

FIG. 9 is a block flow diagram of a process for correcting distortion ina single image or distortion correction and/or co-registration ofmultiple images using artificial landmarks along rib bones of a subject,according to an illustrative embodiment;

FIG. 10 is a block flow diagram of a process for correcting distortionin a single image or distortion correction and/or co-registration ofmultiple images using artificial landmarks along bones of a subject,according to an illustrative embodiment;

FIG. 11 is an image showing automatically detected dangerous regions ofthe image, wherein neighboring rib bones are close to each other,according to an illustrative embodiment;

FIG. 12 is a set of two images showing artificial objects generatedalong a plurality of rib bones within the image before and afterapplication of a volume filter, according to an illustrative embodiment;

FIG. 13 is a block diagram of an example network environment for use inthe methods and systems described herein, according to an illustrativeembodiment; and

FIG. 14 is a block diagram of an example computing device and an examplemobile computing device, for use in illustrative embodiments of theinvention.

FIG. 15 is a series of images recorded at different times, wherein noregistration (e.g. no distortion correction, no co-registration) hasbeen applied to the recorded images, according to an illustrativeembodiment;

FIG. 16 is a series of images recorded at different times, wherein atransformation corresponding to a translation operation has been appliedto each image, according to an illustrative embodiment;

FIG. 17 is a series of images recorded at different times, wherein alinear transformation has been applied to each image, according to anillustrative embodiment;

FIG. 18 is a series of images recorded at different times, wherein atransformation comprising a linear operation followed by a non-linearoperation has been applied to each image, according to an illustrativeembodiment;

FIG. 19 is a series of images showing graphical representations of bonesand aerated regions of lungs of a subject following non-linearregistration of images, according to an illustrative embodiment.

The features and advantages of the present disclosure will become moreapparent from the detailed description set forth below when taken inconjunction with the drawings, in which like reference charactersidentify corresponding elements throughout. In the drawings, likereference numbers generally indicate identical, functionally similar,and/or structurally similar elements.

Definitions

Approximately: As used herein, the term “approximately” or “about,” asapplied to one or more values of interest, refers to a value that issimilar to a stated reference value. In certain embodiments, the term“approximately” or “about” refers to a range of values that fall within25%, 20%, 19%, 18%, 17%, 16%, 15%, 14%, 13%, 12%, 11%, 10%, 9%, 8%, 7%,6%, 5%, 4%, 3%, 2%, 1%, or less in either direction (greater than orless than) of the stated reference value unless otherwise stated orotherwise evident from the context and except where such number wouldexceed 100% of a possible value.

Image: As used herein, an “image”—for example, a 3-D image ofmammal—includes any visual representation, such as a photo, a videoframe, streaming video, as well as any electronic, digital ormathematical analogue of a photo, video frame, or streaming video. Anyapparatus described herein, in certain embodiments, includes a displayfor displaying an image or any other result produced by the processor.Any method described herein, in certain embodiments, includes a step ofdisplaying an image or any other result produced via the method.

3-D, three-dimensional: As used herein, “3-D” or “three-dimensional”with reference to an “image” means conveying information about threedimensions. A 3-D image may be rendered as a dataset in three dimensionsand/or may be displayed as a set of two-dimensional representations, oras a three-dimensional representation. In certain embodiments, a 3-Dimage is represented as voxel (e.g., volumetric pixel) data. As usedherein, the phrase “point of an image” refers to a voxel of the image.

Various medical imaging devices and other 3-D imaging devices (e.g., acomputed tomography scanner (CT scanner), a microCT scanner, etc.)output 3-D images comprising voxels or otherwise have their outputconverted to 3-D images comprising voxels for analysis. In certainembodiments, a voxel corresponds to a unique coordinate in a 3-D image(e.g., a 3-D array). In certain embodiments, each voxel exists in eithera filled or an unfilled state (e.g., binary ON or OFF).

Region of an image: As used herein the term “region” as used in “regionof an image” refers to a collection of points within the image. A regionof an image may be identified as a region that is within specificdistance interval from a reference object of the image. In certainembodiments, the reference object is a specific, single point within theimage and the identified region is a collection of points wherein, foreach point of the region, a distance from the given point to thereference point is within the specific distance interval. In certainembodiments, the reference object is a one-dimensional (1-D) line andthe identified region is a collection of points wherein, for each pointof the region, a shortest distance from the given point to a point ofthe 1-D reference line is within the specific distance interval. Incertain embodiments, the reference object is a two-dimensional (2-D)surface and the identified region is a collection of points wherein, foreach point of the region, a shortest distance from the given point to apoint of the 2-D surface is within the specific distance interval. Incertain embodiments, the reference object is a three-dimensional regionof the image and the identified region is a collection of pointswherein, for each point of the region, a shortest distance from thegiven point to a surface of the reference object is within the specificdistance interval. In certain embodiments, the reference object is athree-dimensional region of the image and the identified region is acollection of points wherein, for each point of the region, a shortestdistance from the given point to a center line of the reference objectis within the specific distance interval.

Mask: As used herein, a “mask” is a graphical pattern that identifies a2-D or 3-D region and is used to control the elimination or retention ofportions of an image or other graphical pattern. In certain embodiments,a mask is represented as a binary 2-D or 3-D image, wherein each pixelof a 2-D image or each voxel of a 3-D image is assigned one of twovalues of a binary set of values (e.g. each pixel or voxel may beassigned a 1 or a 0, e.g. each pixel or voxel may be assigned a Boolean“true” or “false” value).

Registration: As used herein, the terms “registration,” and“registering,” as in registration of one or more images or registeringone or more images refers to transforming an image or images in order toproduce a corresponding image or corresponding images having astandardized view. In certain embodiments registration of an imagecomprises correcting distortions in the image in order to produce asymmetrized version of the image. In certain embodiments, registrationof an image includes co-registering the image with one or more otherimages.

Provide: As used herein, the term “provide”, as in “providing data”,refers to a process for passing data in between different softwareapplications, modules, systems, and/or databases. In certainembodiments, providing data comprises the execution of instructions by aprocess to transfer data in between software applications, or in betweendifferent modules of the same software application. In certainembodiments a software application may provide data to anotherapplication in the form of a file. In certain embodiments an applicationmay provide data to another application on the same processor. Incertain embodiments standard protocols may be used to provide data toapplications on different resources. In certain embodiments a module ina software application may provide data to another module by passingarguments to that module.

DETAILED DESCRIPTION

It is contemplated that systems, architectures, devices, methods, andprocesses of the claimed invention encompass variations and adaptationsdeveloped using information from the embodiments described herein.Adaptation and/or modification of the systems, architectures, devices,methods, and processes described herein may be performed, ascontemplated by this description.

Throughout the description, where articles, devices, systems, andarchitectures are described as having, including, or comprising specificcomponents, or where processes and methods are described as having,including, or comprising specific steps, it is contemplated that,additionally, there are articles, devices, systems, and architectures ofthe present invention that consist essentially of, or consist of, therecited components, and that there are processes and methods accordingto the present invention that consist essentially of, or consist of, therecited processing steps.

It should be understood that the order of steps or order for performingcertain action is immaterial so long as the invention remains operable.Moreover, two or more steps or actions may be conducted simultaneously.

The mention herein of any publication, for example, in the Backgroundsection, is not an admission that the publication serves as prior artwith respect to any of the claims presented herein. The Backgroundsection is presented for purposes of clarity and is not meant as adescription of prior art with respect to any claim.

Documents are incorporated herein by reference as noted. Where there isany discrepancy in the meaning of a particular term, the meaningprovided in the Definition section above is controlling.

Headers are provided for the convenience of the reader—the presenceand/or placement of a header is not intended to limit the scope of thesubject matter described herein.

Described herein are systems and methods for distortion correctionand/or co-registration of 3-D images via automated generation ofartificial landmarks along bones pictured within the 3-D images.

In certain embodiments, the approach described herein facilitatescomparison of 3-D images of a region of a subject that are recorded atdifferent points in time. Comparing multiple 3-D images, wherein eachimage is recorded at a different point in time is relevant formonitoring the progression of diseases such as pulmonary diseases. Incertain embodiments, monitoring disease progression includes measuringtumor growth, fibrosis, edema and combinations thereof over time, basedon data recording in 3-D images. In certain embodiments, changes in themorphology of lung tissue and surrounding connective tissues occur as aphysiological response to the progression of a pulmonary disease, and,accordingly, are monitored over time. For example, in addition tochanges in tumor volume, a researcher or clinician may wish to monitor adownward expansion of a subject's lungs in relation to the rib bones ofthe subject—such changes in lung morphology can be indicative ofcompensation for a reduction of volume and/or functioning of regions ofthe subject's lungs.

Analyzing 3-D images in this manner, by comparing different imagesrecorded at different time points is challenging because for each image,the subject (e.g. a mouse, e.g. a human subject) is often in a differentpose and position. Accordingly, apart from meaningful variations thatare indicative of e.g. changes in tissue morphology (e.g. lung tissue)of a subject, images recorded at different time points also comprisevariations with respect to each other corresponding to distortions andposition differences resulting from pose and position differences of thesubject during recording of each image.

FIG. 1 shows a series of four projections 102, 104, 106, 108 showinggraphical representations of axial skeleton bones of a mouse. Eachprojection is a projection of a 3D mask that identifies axial skeletonbones in a 3-D image. In each projection, the intensity of each pixel isthe average intensity of mask voxels along the third dimension. Eachprojection corresponds to a different 3-D image of a region of themouse, each image, recorded at a different time point. Each of the fourimages to which the displayed projections correspond was recordedapproximately one week after the previous image (e.g. projection 102corresponds to a first image; projection 104 corresponds to a second 3-Dimage recorded approximately one week after the first image; projection106 corresponds to a third image recorded approximately one week afterthe second image; projection 108 corresponds to a fourth image recordedapproximately one week after the third image). The differences in theshape and orientation of the bones of the mouse shown in each projectionillustrate the rather arbitrary pose and/or position that the mouse wasin during recording of each image.

Turning to FIG. 2, in certain embodiments distortion correction isapplied to one or more images of a subject, thereby correcting fordifferences in the pose of the subject at the time a given image wasrecorded. FIG. 2 shows a set 200 of two projections of masks identifyingaxial skeleton bones in a 3-D image of a region of a mouse comprisingthe mouse's ribcage (e.g. a plurality of rib bones) as well as thebackbone. The left projection 210 shows the ribcage bones of the mouse,prior to distortion correction, and the right projection 250 shows theribcage bones of the mouse of the same 3-D image, after distortioncorrection. The portion of the skeleton of the mouse shown in projection210 has an asymmetrical shape that reflects the particular pose of themouse when the 3-D image was recorded. The distortion correctionoperation, when applied to the 3-D image, transforms the image such thatthe skeleton of the mouse has a symmetrical shape, yielding thecorrected 3-D image to which projection 250 corresponds. In particular,each coordinate of the distortion corrected image (to which projection250 corresponds) is related to a corresponding coordinate of theoriginal image (to which projection 210 corresponds) via atransformation (e.g. a distortion field correction) described in thefollowing. Accordingly, distortion correction of a given image comprisesdetermining an appropriate transformation that, when applied to theimage, produces a symmetrized version.

Turning to FIG. 3, in certain embodiments, co-registration is applied toa series of images (e.g. each image having been recorded at a differenttime point). Co-registration aligns multiple images such that the samephysical location is represented by a single set of coordinates in eachimage. That is, for example, a given point P corresponding to aparticular physical location in a subject's lungs is represented by asingle set of coordinates (x₀, y₀, z₀) in each image, as opposed to,e.g. being represented by a first set of coordinates (x₁, y₁, z₁) in afirst image and a second set of coordinates (x₂, y₂, z₂) in a secondimage.

Coordinate values such as x-, y-, and z-coordinates are used throughout,but it should be understood that the specific coordinates (e.g. “x,y,z”)used herein have no special significance in and of themselves, but arerather used for convenience in referring to particular directions andparticular locations in an image, and do not impose any particularrequirement of a coordinate system or approach for measuring distancesin an image for use in the systems and methods described herein. Forexample, it is not necessary that a Cartesian coordinate system be used,and other coordinate systems, such as spherical or cylindrical systems,may be used to describe locations of particular points in an image.

In certain embodiments, the coordinate axes are defined based on theaxes of image acquisition equipment used to record a given 3-D image. Incertain embodiments, x-, y-, and z-coordinate axes are selected based ontheir alignment with particular directions with respect to the body ofthe subject.

For example, in certain embodiments, the x-coordinate axis is selectedas the coordinate axis that is most closely aligned with a directionsubstantially across the body of a subject, from the left side of thesubject to the right side of the subject, or vice versa (e.g. in theopposite direction). Accordingly, in certain embodiments, an x-directioncorresponds to a direction proceeding substantially across the body of asubject, from the left side of the subject to the right side of thesubject, or vice versa (e.g. in the opposite direction).

In certain embodiments, the y-coordinate axis is selected as thecoordinate axis that is most closely aligned with a directionsubstantially from the front side (e.g. chest) to the backside (e.g.spine) of a subject, or vice versa (e.g. in the opposite direction).Accordingly, in certain embodiments, a y-direction corresponds to adirection proceeding from the front side (e.g. chest) to the backside(e.g. spine) of a subject, or vice versa (e.g. in the oppositedirection).

In certain embodiments, the z-coordinate axis is selected as thecoordinate axis that is most closely aligned with a directionsubstantially from the head to tail of a subject, or vice versa (e.g. inthe opposite direction). Accordingly, in certain embodiments, az-direction corresponds to a direction proceeding substantially from thehead to tail of a subject, or vice versa (e.g. in the oppositedirection).

In certain embodiments, a preliminary transformation (e.g. a preliminaryrotation and translation transformation determined using principal axesof the identified backbone and/or principal axes of an identifiedbreastbone in the graphical representation) step is carried out in orderto align coordinate axes with particular directions along the subject,such as those described above. In certain embodiments, transformationsdetermined using the artificial landmarks described herein aligncoordinate axes with particular directions along the subject, such asthose described above.

FIG. 3 shows cross-sections 300 of a series of 3-D images collected atdifferent time points, wherein distortion correction and co-registrationhas been applied to the different 3-D images recorded at different timepoints. Four cross-sections in an xz plane (e.g. slices in the xz plane)304 a, 306 a, 308 a, and 310 a are shown along with cross-sections in anxy plane 304 b, 306 b, 308 b, and 310 b. A cross-section in the zy plane302 is also shown for reference. Cross-sections 304 a and 304 bcorrespond to xz and xy planes, respectively, of an image recorded at afirst time point, t₁. Cross-sections 306 a and 306 b correspond to xzand xy planes, respectively, of an image recorded at a second timepoint, t₂. Cross-sections 308 a and 308 b correspond to xz and xyplanes, respectively, of an image recorded at a third time point, t₃.Cross-sections 310 a and 310 b correspond to xz and xy planes,respectively, of an image recorded at a fourth time point, t₄.Distortion correction and co-registration of images, as shown in FIG. 3facilitates visualization of disease progression (e.g. tumor growth,fibrosis, edema) over time.

A. Registration of Images Via Artificial Landmarks

Turning to FIG. 4, in certain embodiments, co-registration along withdistortion correction is accomplished by applying a transformationcomprising one or more linear and/or nonlinear operations to one or moreimages of a series of images. In certain embodiments, the transformationcorresponds to a linear transformation comprising a translation androtation. The linear transformation maintains the shape of objects asmeasured within the images, but corrects against arbitrary location androtation of the images with respect to each other. In certainembodiments, other linear transformations may include a rigidtransformation (e.g. a translation, e.g. a rotation), an affinetransformation, and any combination thereof.

In certain embodiments, the transformation corresponds to a non-lineartransformation, such as a regularized or smoothed extrapolation ofdistortion field, a thin-plate spline, or any transformation comprisingan extrapolation operation and/or a smoothing operation. In certainembodiments the non-linear transformation comprises a distortion fieldcorrection. In certain embodiments, the non-linear transformationcorrects against distortion (e.g. due to pose differences between theimages). In certain embodiments, the non-linear transformation alsoprovides fine co-registration of the series of images.

In certain embodiments, a linear transformation can also be used fordistortion correction. In certain embodiments registration of an imagecomprises determining and applying a linear transformation as apreliminary step, which facilitates determination of a non-lineartransformation.

FIG. 4 is a set of projections 400 showing representations of ribcagebones of a mouse identified in a 3-D image of the mouse. The set ofprojections 400 illustrates the application of, first, a lineartransformation comprising a rotation and, second, a non-lineartransformation corresponding to a distortion field, according to anillustrative embodiment. In FIG. 4, a first projection 402 shows ribcagebones of the mouse identified from a 3-D image as initially recorded. Asecond projection 404 shows the results of application of a lineartransformation comprising a rotation to the 3-D image. Accordingly, theribcage of the subject as shown in the second projection 404 is rotatedwith respect to the first projection 402. Finally, the result ofapplication of a non-linear transformation for distortion correction isshown in a third projection 406.

Accordingly, in certain embodiments, in order to register an image tocorrect distortions within the image and/or co-register the image withanother image, one or more transformations corresponding to linearand/or non-linear operations are determined and applied to the image.The systems and methods described herein provide for the determinationof artificial landmarks from graphical representations of bones withinan image.

In embodiments described herein, registration (e.g. comprisingdistortion correction and/or co-registration) of images is accomplishedusing artificially generated landmarks within the images. In general,landmarks (also referred to as markers, tags, or control points)correspond to specific points within an image that are identified ascorresponding to a particular physical location of the subject that isimaged. The correspondence between landmarks and particular physicallocations can be leveraged in order to determine appropriatetransformations for distortion correction of a single image as well asdistortion correction and/or co-registration of multiple images.

The artificial landmark-based registration approach described hereincontrasts with previous approaches that accomplish co-registration byperforming a cross-correlation that maximizes mutual information betweentwo images (see, e.g., Maes et al., “Multimodality image registration bymaximization of mutual information,” IEEE Transactions on MedicalImaging 1997, 16 (2): 187-198). Such cross-correlation based approachesdo not use landmarks of any kind, and work best if changes in thecontent of images are absent or marginal.

In contrast to “natural” landmarks, which correspond to naturallyoccurring, known distinguishable objects such as specific individual ribbones, specific joints, vertebrae and the like, artificial landmarks aredetermined from artificial objects (e.g. a given artificial landmark maybe determined as a center of mass of a corresponding artificial object).Artificial objects correspond to fractions of real bones, that aregenerated in silico, via a series of morphological and logicaloperations applied to graphical representations of bones within images,as will be described in the following. Notably, while identification ofnatural objects within an image is sensitive to errors in imageprocessing methods used to identify specific individual bones (e.g.segmentation), generation of artificial objects is robust to imagesegmentation errors and variations in image quality. Moreover, becauseartificial objects are generated via a series of morphological andlogical operations, the density of such objects can be freely varied,which is advantageous to determining image registration transformations.Accordingly, artificial landmarks determined from artificial objectsfacilitate image registration.

In certain embodiments, artificial landmarks are used to determine atransformation used for image registration via a two-stage process.

In certain embodiments, a first stage comprises obtaining (e.g.,determining or receiving) a set of target landmarks to be used alongwith the artificial landmarks in the image in order to determine theimage registration transformation. Each target landmark is associatedwith a matching artificial landmark of the image, and comprises a set oftarget coordinates that represent a desired location of the matchingartificial landmark. The set of target landmarks satisfies certaincriteria, such that alignment of the artificial landmarks with thetarget coordinates yields a desired behavior of the artificiallandmarks. For example, as described in the following, the set targetlandmarks may be symmetric (e.g. substantially invariant under a mirroroperation), such that following alignment of the artificial landmarkswith the target landmarks, the artificial landmarks are themselvessymmetric.

In certain embodiments, in a second stage, an image registrationtransformation is determined using the target landmarks and theartificial landmarks in the given image. In particular, the imageregistration transformation is a transformation that, when applied tocoordinates of the artificial landmarks in the given image, aligns theartificial landmarks with the target landmarks. In certain embodiments,in order to determine the image registration transformation, it is notnecessary to obtain a target landmark for each and every artificiallandmark within an image, and an image registration transformation canbe determined using a subset of the artificial landmarks of the imageand matching target landmarks.

Once determined, the image registration transform can be applied toother points of the image (e.g. not just points corresponding to theartificial landmarks), thereby registering the image. Depending on thetarget landmarks used to determine the image registrationtransformation, the image registration transformation may correctdistortion in (e.g. symmetrize) the given image, and/or co-register thegiven image with one or more other images.

For example, an image registration transformation determined using a setof target landmarks that is symmetric, when applied to points of a givenimage will produce a symmetrized version of the given image, therebycorrecting distortions in the image.

In certain embodiments, a single set of target landmarks is used toregister multiple images. Registering multiple images to the same set oftarget landmarks co-registers the images. If the set of target landmarksis symmetric, then registration of each image using the target landmarkscorrects distortions in each image (e.g. thereby producing symmetrizedversions of each image) as well as co-registers the images.

A.i Obtaining Target Landmarks

Target landmarks can be obtained via multiple approaches. For example,in certain embodiments, a pre-existing set of target landmarks isobtained (e.g. accessed or retrieved from memory by a processor), andused for registration of one or more images.

In certain embodiments, a set of target landmarks is obtained usingartificial landmarks within a given image. For example, a set ofsymmetric target landmarks, which can be used to correct distortion inone or more images, can be determined using artificial landmarks withina given image. In order to determine a set of symmetric targetlandmarks, a mirror operation (e.g. about a predetermined specific plane(e.g. a (e.g. a yz-plane)) is applied to the set of artificial landmarkswithin the given image, and locations of artificial landmarks before andafter the mirror operation are compared (e.g. locations of the originalartificial landmarks are compared with locations of the mirroredartificial landmarks). Comparing locations of the original artificiallandmarks with locations of the mirrored artificial landmarks is used todetermine, for each artificial landmark of at least a portion of theartificial landmarks, a set of target coordinates (e.g. a targetlandmark), such that the set of target landmarks (comprising the targetlandmarks determined for each artificial landmark) is symmetric.

For example, FIG. 5 shows a series of 2-D projections 500 of an image ofa ribcage region of a mouse identifying different artificial landmarksgenerated along a breastbone (as shown in image 502), left ribs (asshown in image 504), and right ribs of a subject (e.g. a mouse) (asshown in image 506). In certain embodiments, locations of artificiallandmarks along left rib bones and right rib bones are compared withmirrored locations of landmarks along left rib bones and right ribbones. A desired behavior in a distortion corrected (e.g. symmetrized)image is that for each artificial landmark on each rib bone, a locationof the artificial landmark is approximately the same as a mirroredlocation of a matching opposite rib bone partner landmark on a matchingopposite rib bone.

In particular, in the embodiments shown in FIG. 5, a chain of artificiallandmarks is generated along each rib bone of the subject. Each rib boneartificial landmark thus corresponds to a particular physical locationthat is on a specific number rib bone (e.g. a first, second, third, etc.rib bone), on a specific side (e.g. a right side or a left side) of thesubject, and is a specific distance along the specific rib bone (e.g.proceeding outwards along the rib bone, away from the backbone of thesubject). Accordingly, a given rib bone artificial landmark can beidentified by a number and side of the specific rib bone on which it isgenerated, and an indication of its distance along the specific rib bone(e.g. a value corresponding to the distance; e.g. an index value thatidentifies its position in the chain along a the rib bone). For a givenrib bone artificial landmark, the opposite rib bone partner to which itis matched is an artificial landmark determined to be approximately asame distance along a same number rib bone, but on the opposite side ofthe subject. Additional information, such as index values, can beassociated (e.g. assigned) to artificial landmarks when they aregenerated, and used to facilitate the process of matching an artificiallandmark to an opposite rib bone partner.

By matching rib bone artificial landmarks to opposite rib bone partnerlandmarks, a plurality of pairs of opposite rib bone artificiallandmarks are identified, each pair comprising a rib bone artificiallandmark and its opposite rib bone partner landmark. The identifiedpairs of opposite rib bone artificial landmarks can be used to determinecoordinates of symmetrical target landmarks. For example, in certainembodiments, for each artificial landmark, coordinates of its targetlandmark are determined as the arithmetic average of the coordinates ofthe artificial landmark and mirrored coordinates of its opposite ribbone partner landmark. In particular, for a given artificial landmarkhaving coordinates (x₁, y₁, z₁) and a matching opposite rib bone partnerlandmark having mirrored coordinates (x₂, y₂, z₂), the arithmeticaverage of the coordinates is determined as ((x₁+x₂)/2, (y₁+y₂)/2,(z₁+z₂)/2). Determining, for each artificial landmark, coordinates oftarget landmarks in this manner generates a set of symmetric targetlandmarks that can be used for distortion correction. In certainembodiments, prior to calculation of the arithmetic averages asdescribed above, a rotation and translation transformation is applied asa preliminary step. In certain embodiments, the preliminary rotation andtranslation transformation is determined using principal axes of anidentified backbone and/or principal axes of an identified breastbone.In certain embodiments, it is not necessary that a target landmark begenerated for every rib bone artificial landmark in the image. Inparticular, in certain embodiments, for a given artificial landmark, itmay not be possible to identify a matching opposite rib bone partnerlandmark. In certain embodiments, when, for a given artificial landmark,a matching opposite rib bone partner landmark is not identified, thegiven artificial landmark is discarded from the set of artificiallandmarks, and not used in registration.

In certain embodiments, a target landmark is also determined for each(or at least a portion) of artificial landmarks along the breastbone. Inparticular, it is desired that breastbone landmarks remain constant uponmirroring of the image. Accordingly, for each breastbone artificiallandmark, a corresponding target landmark is calculated as thearithmetic average of the landmark coordinates and its mirroredcoordinates. In certain embodiments wherein a preliminary translationand rotation operation is determined and applied as described above withregard to rib bone artificial landmarks, the preliminary transformationis also applied to breastbone artificial landmarks (e.g., not only toartificial landmarks on rib bones).

A.ii Determining Image Registration Transformations

Once a set of target landmarks is obtained for use in registration ofone or more images, an image registration transformation is determinedfor each of the one or more images using (i) the set of target landmarksand (ii) landmarks (e.g. artificial landmarks) of each of the one ormore images. For a given image, the image registration transformation isdetermined to substantially optimize alignment of the set of artificiallandmarks of the image with the set of target landmarks.

In certain embodiments, in order to determine a transformation thataligns a set of landmarks (e.g. artificial landmarks) of a given imagewith a set of target landmarks, there is a task to identify the specifictarget landmark to which the given landmark should be aligned. As willbe described in the following, matching an artificial landmark to itsadequate target landmark may be accomplished based on a comparison ofindex values associated with (e.g. assigned to) artificial landmarks andtarget landmarks, and/or coordinates of the artificial and targetlandmarks.

Once landmarks (e.g. artificial landmarks) are matched to targetlandmarks, a transformation that optimizes alignment of the landmarkswith the target landmarks is determined.

For example, in certain embodiments, the transformation is determined bydetermining, for each artificial landmark in the image, a measureddislocation. The measured dislocation for a given artificial landmark isdetermined as the difference between the coordinates that represent theoriginal location of the given landmark and the coordinates of acorresponding target landmark. The transformation determined in thismanner thus takes the form of a distortion field of three components(x-, y-, and z-components) each being a smooth function of imagecoordinates (x, y, and z). In certain embodiments, the distortion fieldis determined from measured dislocations between the artificiallandmarks and their partner target landmarks, via extrapolation and asmoothing operation. FIG. 6 shows an example x-component of thedistortion field as a function of image coordinates.

In certain embodiments, due to use of the smoothing operation, the valueof the distortion field at locations of one or more particularartificial landmarks differ from the measured dislocations determinedfor each of the particular landmarks. FIG. 7 is a graph that shows, foreach artificial landmark of an image, both the measured dislocation inthe x coordinate as well as x-component of the determined distortionfield at the location of the landmark. In certain embodiments,differences between the extrapolated and measured dislocations are usedto estimate (lower limit of) accuracy of co-registration.

In certain embodiments, once determined, an image registrationtransformation can then be applied to points in a region of the image(e.g. not just points corresponding to the artificial landmarks) therebyproducing a transformed image that is corrected for distortion (e.g. asymmetrized image) and/or co-registered with other images. Accordingly,while the transformation is determined via bone landmarks, it providesfor distortion correction and/or co-registration over regions of theimage comprising representations of soft tissue, thereby providing forthe creation of distortion-free, symmetrized images of e.g. lungs andother soft tissue of a subject. Correcting distortion and/orco-registering images in this manner provides for symmetrical,standardized images that can readily be analyzed, interpreted, and/orcompared with other images (e.g. previously recorded images, e.g.previously recorded images that were corrected for distortion similarly,e.g. co-registered images) in a direct fashion, without having toaccount for variations in pose and/or position of the subject.

In certain embodiments, a transformation determined via artificial bonelandmarks in a first image recorded via a first imaging modality (e.g.microCT) is used for distortion correction in a second image recordedusing a different modality (e.g. an optical imaging modality (e.g. FMT),e.g. PET). In certain embodiments, the first and second images arerecorded at substantially the same time, with the subject insubstantially the same pose and position, via a multimodal imagingsystem (e.g. a multimodal imaging system that combines microCT and PETimaging system, e.g. a multimodal imaging system that combines microCTwith FMT imaging). In certain embodiments, the transformation that isdetermined from the artificial bone landmarks in the first image isapplied to the second image. In certain embodiments, there is a knownspatial relationship between the first and second image (e.g. due to aknown different configuration of detectors and/or sources used to recordthe first and second images), and a modified, second transformation isdetermined from the transformation determined via the artificiallandmarks in the first image. The second transformation is then appliedto the second image in order to register the second image (e.g. correctdistortion in the second image and/or co-register the second image withanother image recorded via the second imaging modality).

B. Automated Generation of Artificial Landmarks

B.i Artificial Landmarks Along Rib Bones and the Breastbone

In certain embodiments, the systems and methods described herein providefor automated generation of artificial landmarks used for distortioncorrection and/or co-registration in the manner described above.Artificial landmarks generated via the approaches described herein arerobust to errors in automated segmentation for identification ofspecific bones within an image, and can be refined via outlier filteringapproaches.

In certain embodiments, a set of artificial landmarks within a givenimage is determined based on a set of artificial objects generatedwithin the image. In certain embodiments, the image of the subjectcomprises a plurality of rib bones of the subject, and artificialobjects are generated along each rib bone of the plurality of rib bones.

FIG. 9 is a block flow diagram that shows an example process 900 forgenerating artificial landmarks along rib bones in a 3-D image of aregion of a subject for distortion correction and/or co-registration ofmultiple images. In certain embodiments, the process 900 begins withreceiving a 3-D image of a region of the subject (910), wherein thereceived 3-D image comprises graphical representation of a plurality ofrib bones (e.g. each rib bone of the subject's ribcage, e.g. a portionof the rib bones of the subject) as well as at least a portion of abackbone of the subject. Following receipt of the 3-D image, in a nextstep 920, rib bones and a backbone within the image are identified(920). Bones may be identified in a fully automated fashion (e.g. viasegmentation), or, in certain embodiments, identification of bones mayinclude a user interaction. For example, in certain embodiments boneswithin the image are automatically identified via a local thresholdapproach, after which the ribcage is automatically identified (see,e.g., Ollikainen and Kask, “Method and system for automated detection oftissue interior to mammalian ribcage from an in vivo image,” U.S. Pat.No. 9,192,348), followed by the backbone. In certain embodiments, the3-D image also comprises a breastbone, which is also identified.

After the rib bones and backbone of the subject are identified withinthe 3-D image, in another step 930 in the process 900 a plurality ofseed rib bone artificial object are generated, each seed rib boneartificial object being within a first distance interval from thebackbone in the graphical representation and corresponding to a regionof a rib bone in the graphical representation.

In certain embodiments, seed rib bone artificial objects are generatedby determining a backbone distance map comprising intensity (e.g.numeric) values at each of a plurality of points in three dimensions,each of the intensity values of the backbone distance map correspondingto a distance from a given point in 3-D space to the nearest point ofthe detected backbone in the image. A backbone distance interval mask isgenerated from the backbone distance map to identify points within theimage whose distance to the backbone is within the first distanceinterval (e.g. the distance interval mask is a binary mask, whereinpoints having a distance to the backbone within the first distanceinterval are assigned a first value (e.g. a numeric 1, e.g. a Booleantrue), and other points are assigned a second value (e.g. a numeric 0,e.g. a Boolean false)). Following generation of the backbone distanceinterval mask, a logical AND operation is applied between the backbonedistance interval mask and a mask corresponding to the identified ribbones, resulting in the identification of a plurality of regions of theidentified rib bones that are within the first distance interval fromthe backbone.

FIG. 8 is an image showing artificial objects generated along left andright rib bones in the manner described above, according to anillustrative embodiment. The image shows thirteen seed rib boneartificial objects (840 a, 840 b, 840 c, 840 d, 840 e, 840 f, 840 g, 840h, 840 i, 840 j, 840 k, 8401, 840 m; collectively 840) generated alongleft rib bones of the subject, and thirteen seed rib bone artificialobjects (820 a, 820 b, 820 c, 820 d, 820 e, 820 f, 820 g, 820 h, 820 i,820 j, 820 k, 8201, 820 m; collectively 820) generated along right ribbones of the subject.

In certain embodiments, separately identifying individual rib bones isnot necessary in order to generate seed rib bone artificial objects.Instead, in certain embodiments, a bone mask, corresponding to aplurality of bones in the vicinity of the rib cage, is determined, fromwhich a backbone and, optionally, a breastbone are identified andseparated (e.g. resulting in determination of a breastbone mask and abackbone mask). The remaining bones of the identified correspondprimarily to rib bones, but each individual rib bone need not bedistinguished. Surprisingly, generation of seed rib bone objects iseasier than separation and identification of individual rib bones.

In certain embodiments, following generation of the seed rib boneartificial objects (930), the process 900 continues with generation ofsubsequent rib bone artificial objects along rib bones in the image(940). In particular, for each automatically generated seed rib boneartificial object, a plurality of associated subsequent rib boneartificial objects are generated. In certain embodiments, each seed ribbone artificial object and the plurality of subsequent rib boneartificial objects forms a chain of rib bone artificial objects alongeach rib bone (e.g. the chain of rib bone artificial objects comprisingthe seed rib bone artificial objects and the subsequently generated ribbone artificial objects).

In certain embodiments for a given seed rib bone artificial object, thechain of rib bone artificial objects is generated in a stepwise fashion.In particular, beginning with the seed rib bone artificial object, afirst subsequent rib bone artificial object is generated, a predefineddistance (e.g. a Euclidean distance, e.g. a geodesic distance, along amask corresponding to the identified rib bones) away from the seed ribbone artificial object, in a direction proceeding away from theidentified backbone. A second subsequent rib bone artificial object isthen generated, a predefined distance (e.g. a Euclidean distance, e.g. ageodesic distance, along a mask corresponding to the identified ribbones) away from the first subsequent rib bone artificial object, in adirection proceeding away from the identified backbone. In this manner,new rib bone artificial objects are generated in a stepwise fashion,each newly generated rib bone artificial object proceeding outwards awayfrom the identified backbone. Morphological and logical operationssimilar to those used for generation of seed rib bone artificial object(e.g. determination of distance maps, thresholding operations, dilationoperations, logical operations (e.g. AND, NOT), and combinationsthereof) can be used for generation of subsequent rib bone artificialobjects from each seed rib bone artificial object.

In certain embodiments, the distances separating each rib boneartificial object associated with a given seed rib bone artificialobject (e.g. along a given rib bone) are all the same, such that anequidistant chain of artificial objects associated with the given seedrib bone artificial object is formed (e.g. along the given rib bone).

In certain embodiments, the process of generating new subsequent ribbone artificial objects in this manner proceeds until a newly generatedrib bone artificial object is determined to be within a predefinedthreshold distance from an identified breastbone in the image.

In certain embodiments, additional steps in the process 900 are employedto ensure that, when subsequent rib bone artificial objects associatedwith a given seed rib bone artificial object are generated, rib boneartificial objects are not accidentally generated along a different ribbones (e.g. a rib bone corresponding to the seed rib bone artificialobject and on a nearby rib bone). This may occur if, for example, incertain regions of the image one or more different rib bones are inclose proximity to each other. In order to address this issue, incertain embodiments, the process 900 includes a step of automaticallyidentifying with the image, one or more dangerous regions correspondingto regions in which a distance between a first and second rib bone inthe graphical representation is below a predefined threshold distance.In certain embodiments, such dangerous regions can be identified withoutthe need to separately identify individual rib bones (e.g. distinguishbetween different rib bones), for example, via the use of amorphological closing operation applied to a bone mask that identifies aplurality of rib bones in the graphical representation. An example image1100 showing several identified dangerous regions 1110 a, 1110 b, 1110c, and 1110 d is shown in FIG. 11.

Each time a new subsequent rib bone artificial object associated with agiven seed rib bone artificial object is generated along a rib bone, theprocess determines whether the newly generated artificial object iswithin a predefined threshold distance from one or more previouslyidentified dangerous regions. If the process determines that the newlygenerated rib bone artificial object is within the predefined thresholddistance from a previously identified dangerous region, generation ofsubsequent rib bone artificial objects associated with the given seedrib bone artificial object is terminated.

In certain embodiments, addition to generation of artificial objectsalong rib bones within the image, artificial objects are automaticallygenerated along an identified breastbone of the subject within thegraphical representation in the image.

In particular, in certain embodiments, the graphical representationcomprises at least a portion of the breastbone of the subject, theportion including the lower (e.g. tail-sided) end of the breastbone. Inorder to generate breastbone artificial objects the breastbone isidentified within the graphical representation. The lower end of thebreastbone is also identified within the graphical representation. Next,a seed breastbone artificial object is automatically generated, the seedbreastbone artificial object corresponding to a region of the identifiedbreastbone that is within a second distance interval from the identifiedlower end of the breastbone.

In certain embodiments, the seed breastbone artificial object isgenerated via a set of morphological and logical operations, in a mannersimilar to the approach for generation of the rib bone seed artificialobjects. In particular, in certain embodiments, the lower end of thebreastbone in the graphical representation is identified as pointcorresponding to a minimal z-coordinate of the identified breastbone inthe graphical representation. The breastbone seed artificial object isthen generated via morphological and logical operations (e.g.determination of a breastbone distance interval mask, followed by an ANDoperation) that identify a region of the identified breastbone that iswithin a predefined maximal distance from the point corresponding to theidentified lower end of the breast bone (e.g. for each point of the seedbreastbone artificial object, a distance (e.g. a Euclidean distance,e.g. a distance along a particular coordinate (e.g. a z-coordinate))from the point to a point corresponding to the identified lower end ofthe breastbone is within the second distance interval).

In certain embodiments, a chain of subsequent breastbone artificialobjects associated with the seed breastbone artificial object aregenerated by beginning with the seed breastbone artificial object and,in a stepwise fashion, generating new subsequent breastbone artificialobjects, each newly generated breastbone artificial object proceedingupwards (e.g. a predefined distance from the previously generatedbreastbone artificial object), away from the lower end of thebreastbone.

In certain embodiments, subsequent breastbone artificial objects aregenerated until a newly generated breastbone artificial object crossesan image border or reaches the other end of the identified breastbone,at which point generation of subsequent breastbone artificial objects isterminated. In certain embodiments, a predefined number of breastboneartificial objects is specified, and generation of subsequent breastboneartificial objects is terminated once the predefined number ofbreastbone artificial objects have been generated.

Accordingly, in certain embodiments, a set of artificial objects withinan image can be generated in an automated fashion. In certainembodiments, the set of artificial objects within a given imagecomprises a plurality of artificial objects generated along graphicalrepresentations of rib bones within the image. In certain embodimentsthe set also comprises a plurality of artificial objects generated alongan identified breastbone within the image. In certain embodiments, inorder to provide for distortion correction and/or co-registration ofmultiple images in the manner described above, an artificial landmark isdetermined from each artificial object of the set. In certainembodiments, each artificial landmark is determined as the mass centerof a corresponding artificial object.

In certain embodiments, once a set of artificial objects is determinedfor a given image, target landmarks are obtained (945) and the set ofartificial objects can be used for distortion registration of the image(e.g., distortion correction and/or co-registration) as describedpreviously (950).

In certain embodiments, additional information is associated withartificial objects and/or artificial landmarks that are determinedtherefrom. For example, a given seed rib bone artificial object can beassociated with (e.g. assigned) a series of indices indicating whetherit corresponds to a left or right side rib (e.g. a left-right index), aswell as a number of a given rib bone to which it corresponds (e.g. a ribnumber index).

For example, in certain embodiments, a left-right index is associatedwith a seed rib bone artificial object based on an x-coordinatedetermined from the seed rib bone artificial object. The x-coordinatemay be an average x-coordinate of points of the seed rib bone artificialobject. In certain embodiments, prior to determining an x-coordinatefrom seed rib bone artificial objects for purposes of associatingleft-right index values to the artificial objects, a rotation andtranslation transformation is applied as a preliminary step. In certainembodiments, the preliminary rotation and translation transformation isapplied to ensure that all left rib bones are located on a left side ofthe image, and all right rib bones are located on the right side of theimage. In certain embodiments, the preliminary rotation and translationtransformation is determined using principal axes of an identifiedbackbone in the graphical representation and/or principal axes of anidentified breastbone in the graphical representation.

Subsequent rib bone artificial objects may also be associated withindices, including a left-right index, a seed index, and a sequenceindex. As with seed rib bone artificial objects, a left-right indexassociated with a given subsequent rib bone artificial object identifieswhether the given subsequent rib bone artificial object corresponds to aleft or right side rib. A seed index associated with a given subsequentrib bone artificial object indicates the seed rib bone artificial objectfrom which it was generated. In certain embodiments, wherein subsequentrib bone artificial objects are generated from a seed rib boneartificial as an ordered chain in a stepwise fashion as described above,the sequence index indicates the order in which the subsequent rib boneartificial object was generated.

Indexing of seed rib bone artificial objects and subsequent rib boneartificial objects may be applied during the generation process. Incertain embodiments, indices of artificial objects are updated followingone or more outlier filtering steps as described below, in order toreflect removal of artificial objects corresponding to outliers.

In certain embodiments, artificial landmarks determined from artificialobjects are associated with indices in a similar fashion. A givenartificial landmark determined from a corresponding artificial objectmay be associated (e.g. assigned) the same indices as those of theartificial object from which it is determined. In certain embodiments,an artificial landmark determined from an artificial object isassociated with additional information characterizing the artificialobject from which the artificial landmark was determined. For example, agiven artificial landmark determined from a corresponding artificialobject is associated with a value corresponding to a determined volumeof the artificial object from which the artificial landmark wasdetermined.

In certain embodiments, indices and additional information associatedwith artificial landmarks are used to facilitate matching of landmarks,e.g., to identify opposite rib bone partners or target landmarks, asdescribed above.

In particular, in certain embodiments, rib bone artificial landmarks arematched to opposite rib bone partner landmarks to produce a symmetricset of target landmarks. Matching of artificial landmarks with theircorresponding opposite rib bone partner landmarks may be accomplished byfirst matching seed rib bone artificial landmarks with correspondingopposite rib bone partner seed landmarks. A given seed landmark may bematched with a partner seed landmark on an opposite rib bone using indexvalues, such as a left-right index and a rib number index, that areassigned to seed rib bone artificial landmarks. For example, in order tomatch a first seed artificial landmark to an opposite rib bone partner,a seed rib bone artificial landmark that has a different left-rightindex but a same rib number index is identified.

In certain embodiments, once seed rib bone artificial landmarks arematched with corresponding opposite rib bone partner seed landmarks onopposite side rib bones, subsequent rib bone artificial objects arematched with corresponding opposite rib bone partner landmarks based ontheir association with a particular seed rib bone artificial object fromwhich they were generated (e.g. based on an associated seed index, e.g.based on an associated seed index and an associated left-right index),and their order in the sequence in which they were generated (e.g. basedon an associated sequence index).

In another example, in order to determine an image registrationtransformation that aligns a set of artificial landmarks of a givenimage with a set of target landmarks, each artificial landmark of thegiven image is matched to a partner target landmark.

In particular, in certain embodiments, a set of target landmarks isdetermined using artificial landmarks of a first image, and used to forregistration of a different, second image. In order to register thesecond image, each of at least a portion of the artificial landmarks inthe second image is matched to its target landmark. In certainembodiments, like artificial landmarks, target landmarks are associatedwith index values, which can be used to facilitate matching betweenartificial landmarks and their partner target landmarks. A given targetlandmark may be assigned index values of an artificial landmark fromwhich it is determined. For example, when each target landmark isdetermined from a particular artificial landmark of the first image, itis assigned the index values of the particular artificial landmark fromwhich it was generated. Similarly, in certain embodiments, a targetlandmark determined from a particular artificial landmark may also beassociated with additional information that is associated with theparticular artificial landmark from which it was determined. Suchadditional information may include a volume of the artificial objectfrom which the particular artificial landmark was determined.Accordingly, artificial landmarks of the second image can be matched topartner target landmarks using, alone or in combination: (i) indexvalues, (ii) additional information, as well as (iii) coordinates of theartificial landmarks of the second image and the target landmarks.

In certain embodiments, artificial landmarks of the second image arematched to target landmarks by first matching seed rib bone artificiallandmarks to target landmarks that were determined from seed rib boneartificial landmarks of the first image. Seed rib bone artificiallandmarks in the second image may be matched to target landmarks using(i) z-coordinates of the artificial landmarks in the second image andthe target landmarks and/or (ii) the indices (e.g. left-right indices,e.g. rib number indices) of the artificial landmarks in the second imageand the indices of the target landmarks. Following matching of seedartificial landmarks with their target landmarks, subsequent rib boneartificial landmarks are matched with corresponding target landmarksbased on their association with a particular seed rib bone artificiallandmark from which they were generated (e.g. based on an associatedseed index, e.g. based on an associated seed index and an associatedleft-right index), and their order in the sequence in which they weregenerated (e.g. based on an associated sequence index).

B.ii Outlier Filtering for Rib Bone and Breastbone Artificial Objects

In certain embodiments, generation of artificial objects (e.g. rib boneartificial objects, e.g. breastbone artificial objects) comprisesadditional processing steps corresponding to filtering steps in order toensure that the automatically generated artificial objects satisfydifferent sets of criteria.

For example, in certain embodiments, a volume filter is applied toeliminate one or more artificial objects corresponding to regionsdetermined to represent a volume below a predefined threshold volume.This approach can be applied to eliminate artifacts during thegeneration of artificial objects, as shown, for example, in FIG. 12.

A first image 1210 of FIG. 12 displays a graphical representation of aplurality of rib bones and a backbone, along with a prospective set ofseed rib bone artificial objects generated by, for example, the process900 described with respect to FIG. 9. As is evident in the image, incertain embodiments, a series of artificial objects with similar volumesis generated, along with several artificial objects having noticeablysmaller volumes. These small volume artificial objects correspond toartifacts (e.g. resulting from noise in the image, e.g. resulting fromerrors in segmentation, e.g. resulting from the presence of physical,non-bone objects, such as grains of sand).

Accordingly, in order to eliminate such artifacts, and ensure thatappropriate seed rib bone artificial objects are generated, followinggeneration of the set of prospective seed rib bone artificial objects avolume filter is applied to the set of prospective seed rib boneartificial object. Application of the volume filter eliminates, from theset, those artificial objects determined to represent a volume below apredefined threshold volume. Following application of the volume filter,seed rib bone artificial objects are selected from the set ofprospective seed rib bone artificial objects. A second image 1250 ofFIG. 12 shows the results of applying a volume filter to the set ofprospective seed rib bone artificial objects shown in the first image1210. The set of artificial objects in the second image 1250 no longerincludes the artificial objects corresponding to artifacts from thefirst image 1210.

In certain embodiments, a volume filter is applied to eliminateartifacts created during the generation of other artificial objects(e.g. any artificial objects, not just the seed rib bone artificialobjects).

In certain embodiments, additional filtering criteria is applied toartificial objects generated along rib bones.

For example, in certain embodiments, a number of rib bones an imagedsubject has is known. Accordingly, in certain embodiments, a number ofseed rib bone artificial objects that has been generated is determined,and compared with a known (expected) number of rib bones that the imagedsubject has. For example, mice have 13 rib bones on each side—13 leftrib bones and 13 right rib bones—accordingly, 26 total rib bones.Accordingly, for an image of a mouse, it is expected that 13 left seedrib bone artificial objects and 13 right seed rib bone artificialobjects will be generated. Therefore, by comparing the number ofgenerated seed rib bone artificial objects with a known (expected)number of ribs that the subject has, erroneously generated artificialobjects corresponding to artifacts can be identified and eliminated.

In certain embodiments, criteria for filtering out artifacts areestablished based on the symmetry of a physical ribcage, and applied tothe set of rib bone artificial objects. In particular, in certainembodiments, in order to apply filtering approaches that leverage thesymmetry of a physical ribcage, rib bone artificial objects are dividedinto left and right rib bone artificial objects, and grouped as pairs ofcorresponding left and right rib bone artificial objects. Pairs ofartificial objects can be determined via a matching process similar tothe process described above for matching artificial landmarks withopposite rib bone partner landmarks for purposes of distortioncorrection. In particular, pairs of corresponding artificial objects onopposite rib bones can be determined using indices associated with theartificial objects (e.g. a left-right index, e.g. a rib number index,e.g. a sequence index). In certain embodiments, coordinates associatedwith the artificial objects (e.g. an average x-coordinate of points ofeach artificial object; e.g. an average z-coordinate of points of eachartificial object) as well as additional information, such as volumes ofartificial objects is used to facilitate determining pairs of artificialobjects on opposite rib bones.

Each pair of rib bone artificial objects thus comprises a first rib boneartificial object associated with a first rib bone (e.g. a right ribbone) of a pair of opposite rib bones and a second rib bone artificialobject associated with a second rib bone (e.g. a left rib bone) of thepair of opposite rib bones. For each pair of rib bone artificialobjects, the first and second rib bone artificial objects of a givenpair should represent a similar volume and have similar heights. Heightof an artificial object may be determined as a thickness in thez-direction of the artificial object, or as a distance along thebackbone (e.g. determined as an average (mean) value of z-coordinates ofpoints of the artificial object). Accordingly, in certain embodiments, apair-wise comparison filter is applied that ensures, for each pair ofrib bone artificial objects, (i) a difference between a volumerepresented by the first object of the pair and a volume represented bythe second object of the pair is below a predefined threshold volumedifference, and/or (ii) a difference between a height of the firstobject of the pair and a height of the second object of the pair isbelow a predefined threshold height difference.

In certain embodiments, the pair comparison filter is applied to removeartifacts during generation of seed rib bone artificial objects. Forexample, following generation of a set of prospective seed rib boneartificial objects (e.g. via the process 900 described above withrespect to FIG. 9), one or more pairs of prospective seed rib boneartificial objects are automatically identified. Each pair comprises afirst seed rib bone artificial object of a first rib bone of the pair ofrib bones and corresponding second seed rib bone artificial object ofthe opposite rib bone (e.g. the first seed rib bone artificial object isalong a right rib and the second seed rib bone artificial object isalong the corresponding left rib).

A pair comparison filter is then applied to the set of prospective seedrib bone artificial objects, wherein application of the pair comparisonfilter eliminates, from the set, pairs of artificial objects for which(i) a difference between a volume represented by the first object of thepair and a volume represented by the second object of the pair is abovea predefined threshold volume difference, and/or (ii) a differencebetween in a height of the first object of the pair and a height of thesecond object of the pair is above a predefined threshold heightdifference. Following application of the pair comparison filter, seedrib bone artificial objects are selected from the set of prospectiveseed rib bone artificial objects.

In certain embodiments, a pair comparison filter is applied to eliminateartifacts created during the generation of other artificial objectsalong rib bones (e.g. subsequent rib bone artificial objects, not justthe seed rib bone artificial objects).

In certain embodiments, another filtering step is applied, comprisingverifying whether the distance between consecutive seed rib boneartificial objects is consistent (along each rib/breastbone). Forexample, the distance (e.g. in the z-direction) between a given seed ribbone artificial object a next seed rib bone artificial object (e gimmediately below and/or above, on the same side of the rib cage) isapproximately constant (e.g. varies slowly) if seed rib bone artificialobjects are correctly generated. Variations in the distance betweenadjacent seed rib bone artificial objects that are above a predefinedthreshold can be identified and used to filter outliers, for example ifa seed rib bone artificial object is generated in a region outside of anactual rib bone.

In certain embodiments, a filtering approach based on comparison ofartificial objects between multiple images is applied. In particular, incertain embodiments artificial objects of a first set of artificialobjects within a first image are compared with artificial objects of asecond set of artificial objects within a second image.

In certain embodiments, filtering based on comparison of artificialobjects between multiple images comprises identifying a plurality ofcross-image pairs of artificial objects, wherein each cross-image paircomprises a first artificial object of a first set of artificial objectsof the first image and a corresponding second artificial object of asecond set of artificial objects of the second image.

In certain embodiments cross image pairs of artificial objects areidentified, automatically, using indices (e.g. left-right indices, e.g.rib number indices, e.g. seed indices, e.g. sequence indices) similar tothe manner described above with respect to matching of artificiallandmarks in a second image with target landmarks determined fromartificial landmarks in a different, first image. In certainembodiments, identification of cross-image pairs of rib bone artificialobjects comprises first identifying, for a given seed rib boneartificial object of the first image, a corresponding seed rib boneartificial object of the second image. In certain embodiments, indices(e.g. left-right indices, e.g. rib number indices) associated with seedrib bone artificial objects are used. In certain embodimentscorresponding seed rib bone artificial objects can be determined bymatching (e.g. searching for a closest match in) z-coordinates. Incertain embodiments, there is an unknown shift in coordinates betweenthe first and second image that is determined and corrected for beforematching z-coordinates. The coordinate shift may be determined via acorrelation technique. For example, a cross-correlation between a (i) afirst set of points identifying positions of the artificial objects ofthe first image and (ii) a second set of points identifying artificialobjects of the second image can be used to determine a shift inz-coordinates between the first and second images. The determined shiftcan then be used to correct for the shift when matching z-coordinates.

Once a given cross-image pair of seed rib bone artificial objects isidentified, subsequent rib bone artificial objects associated with eachseed rib bone artificial object of the cross-image pair of seed rib boneartificial objects are matched (e.g. using an order in which thesubsequent rib bone artificial objects were generated, e.g. using asequence index associate with each associated subsequent rib boneartificial object).

In certain embodiments, the corresponding first and second artificialobjects of a given cross-image pair are compared to determine whether ornot they represent similar volumes. Cross-image pairs for which thefirst and second artificial objects represent significantly differentvolumes are presumed to correspond to artifacts, and, accordinglyeliminated. For example, in certain embodiments, for each cross-imagepair of artificial objects, a difference between a volume of the firstartificial object and a volume of the second artificial object isdetermined. For each cross-image pair, the determined volume differenceis compared to a threshold difference (e.g. a predefined thresholddifference), and, artificial objects of cross-image pairs for which thedetermined volume difference is above the threshold difference areeliminated from the respective set of artificial objects of eachrespective image.

In certain embodiments, for each of a first and second artificial objectof a given cross image pair, information (e.g. values) about locationsof the artificial objects with respect to neighboring artificial objectsis determined, and compared. For example, for the first artificialobject of the cross-image pair, values such as (i) a distance to anearest neighboring artificial object, (ii) an average number ofartificial objects within a predetermined threshold distance from thefirst artificial object, and the like, may be determined. The samevalues may be determined for the second artificial object, and comparedwith the corresponding values determined for the first artificialobject. If differences between values (e.g. a distance to a certainother artificial object) is above a set threshold value, the artificialobjects of the cross image pair are eliminated.

In certain embodiments, any of the above described filtering approaches(e.g. volume filter, pair comparison filter, consecutive objectseparation filter, cross-image pair comparison) are applied incombination, and in various orders. For example, for generation of theseed rib bone artificial objects shown in image 800 of FIG. 8, a volumefilter, pair comparison filter, and consecutive object separation filterwere applied. In certain embodiments, any of the above describedfiltering approaches (e.g. volume filter, pair comparison filter,consecutive object separation filter, cross-image pair comparison) areapplied in combination with each other, as well as other filteringapproaches, and in various orders.

In certain embodiments, any of the operations described herein as beingperformed on an identified volume is the same as an operation performedon a landmark (e.g. a coordinate) associated with the volume.Accordingly, operations such as the above described outlier filteringoperations performed on artificial objects can also be performed onartificial landmarks associated with the artificial objects.

Performing particular operations, such as outlier filtering, onartificial landmarks can be advantageous from the standpoint ofcomputational speed. In particular, while an object is represented by amask in three dimensions, landmarks correspond points represented bycoordinates. In certain embodiments, each landmark is associated withthe artificial object from which it was determined. In certainembodiments, additional information is associated with a given landmark.The additional information associated with a given landmark includesinformation such as a volume of the object from which it is derived,indices such as a left-right index, a rib number index, a sequence indexand the like. Accordingly, each artificial landmark is associated withinformation that can be used to perform the same outlier filtering stepsdescribed above on landmarks, as opposed to artificial object.

In certain embodiments, each artificial landmark is represented as a rowin a table. Columns of the table correspond to properties of eachlandmark, such as an x-coordinate, a y-coordinate, a z-coordinate, aswell as additional information such as a volume of an artificial objectfrom which the landmark was determined, and various indices associatedwith the landmark. Operations on landmarks (each represented by a row ina table) are thus very fast compared to any operation applied to imagesbecause the number of voxels in an image may be many orders of magnitudebigger than the number of entries in a table of landmarks.

B.iii Artificial Landmarks Along Arbitrary Bones

In certain embodiments, artificial objects are generated along variousdifferent bones (e.g. not just rib bones and/or a breastbone of asubject) for distortion correction and/or co-registration.

FIG. 10 shows a series of steps in an example process 1000 forgeneration of artificial objects along one or more bones of interestrepresented in a 3-D image of a region of a subject. In certainembodiments, the process 1000 begins with receiving a 3-D image of aregion of the subject (1010), wherein the 3-D image comprises agraphical representation of one or more bones of interest. The bones ofinterest in the image can include bones of the subject's axial skeleton,such as rib bones, the breastbone, backbone, as well as other types ofbones, from other portions of the skeleton of the subject.

Following receipt of the 3-D image, in a next step 1020, one or morebones of interest are identified (e.g. automatically (e.g. viasegmentation, e.g. via a local threshold approach), e.g. via a userinteraction) within the image. In addition, a reference object withinthe image is also identified (1020). The reference object may be asingle point of the image, a 1-D line, a 2-D surface, or a 3-D region ofthe image. For example, in certain embodiments, a centerline of aparticular bone or portion of a bone is identified within the image andused as a reference object. In certain embodiments, a 2-D surface of abone identified within the image is used as a reference object.

The reference object identified within the image is used as a basis forgenerating seed artificial objects along each of the bones of interest.In particular, in certain embodiments, at step 1030 the process 1000automatically generates, one or more seed artificial objectscorresponding to a regions of the identified bones of interest that arewithin a specific distance interval from the reference object.

In certain embodiments, seed artificial objects are generated by firstdetermining a distance map comprising intensity values (e.g. numericvalues) at each of a plurality of points in three dimensions. Each ofthe intensity values at each point corresponds to a distance from thegiven point to the reference object. The distance map is then used togenerate a distance interval mask that identifies those points in 3-Dspace that are within a specific distance interval from the referenceobject (e.g. the distance interval mask is a binary mask, wherein pointshaving a distance to the reference object within a predefined distanceinterval are assigned a first value (e.g. a numeric 1, e.g. a Booleantrue), and other points are assigned a second value (e.g. a numeric 0,e.g. a Boolean false)).

The distance interval mask can then be used to identify regions of theone or more bones of interest that are within a specific distance fromthe reference object. In certain embodiments, the distance interval maskis used in combination with a mask that identifies the one or more bonesof interest. A logical AND operation is applied (e.g. in a pointwisefashion) between the distance interval mask and the mask correspondingto the identified one or more bones of interest, thereby automaticallyidentifying regions of the bones of interest that are within thedistance interval from the reference object.

In certain embodiments, following generation of the seed artificialobjects (1030), the process 1000 continues with generation of subsequentartificial objects associated with each seed artificial object (1040).In particular, beginning with a given seed artificial objectcorresponding to a region of a bone of interest, a plurality ofassociated subsequent rib bone artificial objects are automaticallygenerated along the bone of interest. In certain embodiments, theartificial objects generated along a bone of interest (e.g. the seedartificial object and the subsequent artificial objects) form a chain ofartificial objects along the bone of interest.

In certain embodiments, for a given seed artificial object, theassociated subsequent artificial objects are generated in a stepwisefashion. For example, beginning with the seed artificial object, a firstsubsequent artificial object is generated along a bone of interest, apredefined distance away from the seed artificial object, in a directionproceeding away from the identified reference object. A secondsubsequent artificial object is then generated along the bone ofinterest, a predefined distance away from the first subsequentartificial object, in a direction proceeding away from the identifiedreference object. In this manner, new artificial objects are generatedin a stepwise fashion, each newly generated artificial object proceedingoutwards away from the identified reference object and along the bone ofinterest.

In certain embodiments, the distances separating each artificial objectalong a given bone of interest are all the same, such that anequidistant chain of artificial objects is formed along the bone ofinterest.

In certain embodiments, a set of morphological and logical operationsare used for generation of subsequent artificial object along each boneof interest.

In certain embodiments, the process 1000 includes steps to ensure thatsubsequent artificial objects generated from a specific seed artificialobject corresponding to a region of a bone of interest are accuratelygenerated along that bone of interest, and not generated along otherbones identified in the image (e.g. the automated processing does notcause subsequent artificial objects to jump to other bones).Accordingly, the process of generating new subsequent artificial objectsmay proceed until a newly generated artificial object is determined tobe within a predefined threshold distance from an identified additionalbone (e.g. not corresponding to one of the one or more identified bonesof interest) within the image. When a newly generated artificial objectis determined to be within a threshold distance of the additional bone,generation of artificial objects along the particular bone of interestis terminated.

In certain embodiments, additional steps in the process 1000 areemployed to ensure that, when subsequent artificial objects associatedwith a given seed artificial object are generated, artificial objectsare not accidentally generated along a multiple different bones ofinterest (e.g. a bone of interest on which the seed artificial object isgenerated and then on a nearby bone of interest). This may occur if, forexample, in certain regions of the image one or more different bones ofinterest are in close proximity to each other. In order to address thisissue, in certain embodiments, the process 1000 includes a step ofautomatically identifying, with the image, one or more dangerous regionscorresponding to regions in which a distance between a first and secondbone of interest is below a predefined threshold distance.

Each time a new subsequent artificial object associated with a givenseed artificial object is generated, the process determines whether thenewly generated artificial object is within a predefined thresholddistance from one or more previously identified dangerous regions. Ifthe process determines that the newly generated artificial object iswithin the predefined threshold distance from a previously identifieddangerous region, generation of subsequent artificial objects associatedwith the given seed artificial object is terminated.

Accordingly, in certain embodiments, a set of artificial objects withinan image can be generated in an automated fashion. In certainembodiments, the set of artificial objects within a given imagecomprises a plurality of artificial objects generated along eachidentified bone of interest within the image.

In certain embodiments, in order to provide for distortion correctionand/or co-registration of multiple images in the manner described above,an artificial landmark is determined from each artificial object of theset. In certain embodiments, each artificial landmarks is determined asthe mass center of a corresponding artificial object.

In certain embodiments, once a set of artificial objects is determinedfor a given image, target landmarks are obtained (1045) and the set ofartificial objects can be used for distortion registration of the image(e.g., distortion correction and/or co-registration) as describedpreviously (1050).

B.iv Outlier Filtering for Artificial Objects Generated Along ArbitraryBones of Interest

In certain embodiments, similar to the approach described for generationof artificial objects along rib bones and/or a breastbone of a subject,generation of artificial objects along various bones of interestcomprises additional processing steps corresponding to filtering stepsin order to ensure that the automatically generated artificial objectssatisfy different sets of criteria.

For example, a volume filter can be applied in a general manner, eachtime one or more artificial objects (seed artificial objects, as well assubsequent artificial objects) are generated, in order to eliminate oneor more artificial objects corresponding to regions determined torepresent a volume below a predefined threshold volume. Excessivelysmall artificial objects, which are identified and eliminated viaapplication of the volume filter, often correspond to artifacts as aresult of errors in image segmentation and/or noise.

In certain embodiments, generation of one or more artificial objectsincludes a first step of generation of a set of prospective artificialobjects. Such prospective artificial objects may be generated via any ofthe approaches described herein (e.g. via a logical AND operationbetween determined masks). In certain cases, this set of prospectiveartificial objects includes extremely small artificial objects, thatcorrespond to artifacts that result from noise and/or image segmentationerrors. In order to eliminate such artifacts, a volume filter is appliedto eliminate, from the set, those artificial objects determined, by theprocessor, to represent a volume below a predefined threshold volume.Following application of the volume filter, each of the desiredartificial object(s) (e.g. each seed artificial object, e.g. a newlygenerated subsequent artificial object associated with a given seedartificial object) is(are) selected from the set (from which the volumefilter has eliminated excessively small artificial objects).

In certain embodiments, other filters that leverage physical intuition,such as a symmetry of particular skeletal regions, may also be applied.In certain embodiments, another filtering step is applied, comprisingverifying whether the distance between consecutive artificial objects isconsistent.

In certain embodiments, a filtering approach based on comparison ofartificial objects between multiple images, based on comparingcross-image pairs is applied as described above.

In certain embodiments, filtering approaches, including, but not limitedto those described above (e.g. volume filter, pair comparison filter,consecutive object separation filter, cross-image pair comparison,filtering approaches based on symmetry of particular skeletal regions)are applied in combination, and in various orders.

C. Examples

FIGS. 15-18 show examples of image registration using the artificiallandmark generation approach described herein. FIG. 15 shows crosssections 1500 of a series of 3-D images of a mouse collected atdifferent time points, without registration. Cross-sections (planes) offour different images are shown, wherein for each image the position andpose of the mouse (subject) are different. Four cross-sections in an xzplane (e.g. slices in the xz plane) 1504 a, 1506 a, 1508 a, and 1510 aare shown along with cross-sections in an xy plane 1504 b, 1506 b, 1508b, and 1510 b. A cross-section in the zy plane 1502 is shown forreference. Cross-sections 1504 a and 1504 b correspond to slices throughxz and xy planes, respectively, of a first image recorded at a firsttime point. Cross-sections 1506 a and 1506 b slices through xz and xyplanes, respectively, of an second image recorded at a second timepoint. Cross-sections 1508 a and 1508 b correspond to xz and xy planes,respectively, of a third image recorded at a third time point.Cross-sections 1510 a and 1510 b correspond xz and xy planes,respectively, of a fourth image recorded at a fourth time point. Theimages in FIG. 15 correspond to raw images, without any distortioncorrection or co-registration applied. Differences in the pose andposition of the mouse in the different measurements are reflected in theimages, as shown in the 2-D cross sections of the figure.

FIG. 16 shows an example of determining transformations corresponding totranslation operations and applying the transformations to the 3-Dimages. In order to determine the transformations, artificial landmarkswere generated for each image in a series of images using the approachdescribed herein. A set of target landmarks was determined using theartificial landmarks in one of the images. The target landmarks weredetermined to be symmetrical under a mirror operation about a yz-plane.For each image in the series, a corresponding transformationcorresponding to a translation operation was determined to align the setof artificial landmarks in the respective image with the set of targetlandmarks.

The determined translation operation was applied to each correspondingimage, and the results are shown in the cross-sections 1600 shown inFIG. 16. Analogous to FIG. 15, cross-sections 1604 a and 1604 bcorrespond to xz and xy planes, respectively, of a first imagecorresponding to a first measurement recorded at a first time point.Cross-sections 1606 a and 1606 b correspond to xz and xy planes,respectively, of a second image recorded at a second time point.Cross-sections 1608 a and 1608 b correspond to xz and xy planes,respectively, of a third image recorded at a third time point. 1610 aand 1610 b correspond to xz and xy planes, respectively, of a fourthimage corresponding to a fourth measurement. A cross-section in the yzplane 1602 is also shown. As a result of application of a determinedtranslation operation to each image in the series, a mean coordinate ofartificial landmarks of each image is the same for all four images.Where correction is needed for rotation of the subject with respect tothe axes of the image acquisition instrument, the transformationapproach of either FIG. 17 or FIG. 18 may be superior.

FIG. 17 shows an example wherein for each image in the series of fourimages, a transformation corresponding to a linear transformation wasdetermined using artificial landmarks in the image, and applied to theimage. Artificial landmarks were generated for each image in the series,and symmetrical target landmarks were determined using one of the imagesin the series as previously described. For each image, a correspondinglinear transformation was determined using the artificial landmarks inthe respective image and the target landmarks (e.g. the lineartransformation was determined to substantially optimize alignment of theset of artificial landmarks of a given image with the set of targetlandmarks).

The results of application of, for each image, the correspondingdetermined linear transformation to the image, are shown in the seriesof cross-sections 1700 shown in FIG. 17. Analogous to FIGS. 15 and 16,cross-sections 1704 a and 1704 b correspond to xz and xy planes,respectively, of a first image recorded at a first time point.Cross-sections 1706 a and 1706 b correspond to xz and xy planes,respectively, of an second image recorded at a second time point.Cross-sections 1708 a and 1708 b correspond to xz and xy planes,respectively, of a third image recorded at third time point.Cross-sections 1710 a and 1710 b correspond to xz and xy planes,respectively, of a fourth image corresponding to a fourth measurement. Across-section in the yz plane 1702 is also shown.

FIG. 18 shows an example wherein for each image in the series of fourimages, a transformation comprising a linear transformation followed bya non-linear transformation was determined using artificial landmarks inthe image, and applied to the image. Artificial landmarks were generatedfor each image in the series, and symmetrical target landmarks weredetermined using one of the images in the series as previouslydescribed. For each image, a combined transformation (corresponding to alinear transformation followed by a non-linear transformation) wasdetermined using the artificial landmarks in the respective image andthe target landmarks (e.g. transformation was determined tosubstantially optimize alignment of the set of artificial landmarks of agiven image with the set of target landmarks).

The results of application of, for each image, the correspondingdetermined transformation (linear transformation followed by anon-linear transformation) to the image, are shown in the series ofcross sections 1800 shown in FIG. 18. Analogous to FIGS. 15-17,cross-sections 1804 a and 1804 b correspond to xz and xy planes,respectively, of a first image recorded at a first time point.Cross-sections 1806 a and 1806 b correspond to xz and xy planes,respectively, of an second image recorded at a second time point.Cross-sections 1808 a and 1808 b correspond to xz and xy planes,respectively, of a third image recorded at a third time-point.Cross-sections 1810 a and 1810 b correspond to xz and xy planes,respectively, of a fourth image recorded at a fourth time point. Across-section in the yz plane 1802 is also shown.

FIG. 19 shows a series of projections 1910, 1920, 1930, 1940 ofidentified ribcage bones (e.g. a bone mask) and identified aeratedlungs, each projection representing ribcage bones and aerated lungsidentified within a different 3-D image of a mouse, recorded at adifferent time. The different 3-D images have been co-registered, and,accordingly, can readily be compared to observe variations in thesubject's lungs.

D. Network Environment and Computing System

As shown in FIG. 13, an implementation of a network environment 1300 foruse in providing systems, methods, and architectures for generation ofartificial landmarks for distortion correction and/or co-registration ofimages as described herein. In brief overview, referring now to FIG. 13,a block diagram of an exemplary cloud computing environment 1300 isshown and described. The cloud computing environment 1300 may includeone or more resource providers 1302 a, 1302 b, 1302 c (collectively,1302). Each resource provider 1302 may include computing resources. Insome implementations, computing resources may include any hardwareand/or software used to process data. For example, computing resourcesmay include hardware and/or software capable of executing algorithms,computer programs, and/or computer applications. In someimplementations, exemplary computing resources may include applicationservers and/or databases with storage and retrieval capabilities. Eachresource provider 1302 may be connected to any other resource provider1302 in the cloud computing environment 1300. In some implementations,the resource providers 1302 may be connected over a computer network1308. Each resource provider 1302 may be connected to one or morecomputing device 1304 a, 1304 b, 1304 c (collectively, 1304), over thecomputer network 1308.

The cloud computing environment 1300 may include a resource manager1306. The resource manager 1306 may be connected to the resourceproviders 1302 and the computing devices 1304 over the computer network1308. In some implementations, the resource manager 1306 may facilitatethe provision of computing resources by one or more resource providers1302 to one or more computing devices 1304. The resource manager 1306may receive a request for a computing resource from a particularcomputing device 1304. The resource manager 1306 may identify one ormore resource providers 1302 capable of providing the computing resourcerequested by the computing device 1304. The resource manager 1306 mayselect a resource provider 1302 to provide the computing resource. Theresource manager 1306 may facilitate a connection between the resourceprovider 1302 and a particular computing device 1304. In someimplementations, the resource manager 1306 may establish a connectionbetween a particular resource provider 1302 and a particular computingdevice 1304. In some implementations, the resource manager 1306 mayredirect a particular computing device 1304 to a particular resourceprovider 1302 with the requested computing resource.

FIG. 14 shows an example of a computing device 1400 and a mobilecomputing device 1450 that can be used to implement the techniquesdescribed in this disclosure. The computing device 1400 is intended torepresent various forms of digital computers, such as laptops, desktops,workstations, personal digital assistants, servers, blade servers,mainframes, and other appropriate computers. The mobile computing device1450 is intended to represent various forms of mobile devices, such aspersonal digital assistants, cellular telephones, smart-phones, andother similar computing devices. The components shown here, theirconnections and relationships, and their functions, are meant to beexamples only, and are not meant to be limiting.

The computing device 1400 includes a processor 1402, a memory 1404, astorage device 1406, a high-speed interface 1408 connecting to thememory 1404 and multiple high-speed expansion ports 1410, and alow-speed interface 1412 connecting to a low-speed expansion port 1414and the storage device 1406. Each of the processor 1402, the memory1404, the storage device 1406, the high-speed interface 1408, thehigh-speed expansion ports 1410, and the low-speed interface 1412, areinterconnected using various busses, and may be mounted on a commonmotherboard or in other manners as appropriate. The processor 1402 canprocess instructions for execution within the computing device 1400,including instructions stored in the memory 1404 or on the storagedevice 1406 to display graphical information for a GUI on an externalinput/output device, such as a display 1416 coupled to the high-speedinterface 1408. In other implementations, multiple processors and/ormultiple buses may be used, as appropriate, along with multiple memoriesand types of memory. Also, multiple computing devices may be connected,with each device providing portions of the necessary operations (e.g.,as a server bank, a group of blade servers, or a multi-processorsystem). Thus, as the term is used herein, where a plurality offunctions are described as being performed by “a processor”, thisencompasses embodiments wherein the plurality of functions are performedby any number of processors (one or more) of any number of computingdevices (one or more). Furthermore, where a function is described asbeing performed by “a processor”, this encompasses embodiments whereinthe function is performed by any number of processors (one or more) ofany number of computing devices (one or more) (e.g., in a distributedcomputing system).

The memory 1404 stores information within the computing device 1400. Insome implementations, the memory 1404 is a volatile memory unit orunits. In some implementations, the memory 1404 is a non-volatile memoryunit or units. The memory 1404 may also be another form ofcomputer-readable medium, such as a magnetic or optical disk.

The storage device 1406 is capable of providing mass storage for thecomputing device 1400. In some implementations, the storage device 1406may be or contain a computer-readable medium, such as a floppy diskdevice, a hard disk device, an optical disk device, or a tape device, aflash memory or other similar solid state memory device, or an array ofdevices, including devices in a storage area network or otherconfigurations. Instructions can be stored in an information carrier.The instructions, when executed by one or more processing devices (forexample, processor 1402), perform one or more methods, such as thosedescribed above. The instructions can also be stored by one or morestorage devices such as computer- or machine-readable mediums (forexample, the memory 1404, the storage device 1406, or memory on theprocessor 1402).

The high-speed interface 1408 manages bandwidth-intensive operations forthe computing device 1400, while the low-speed interface 1412 manageslower bandwidth-intensive operations. Such allocation of functions is anexample only. In some implementations, the high-speed interface 1408 iscoupled to the memory 1404, the display 1416 (e.g., through a graphicsprocessor or accelerator), and to the high-speed expansion ports 1410,which may accept various expansion cards (not shown). In theimplementation, the low-speed interface 1412 is coupled to the storagedevice 1406 and the low-speed expansion port 1414. The low-speedexpansion port 1414, which may include various communication ports(e.g., USB, Bluetooth®, Ethernet, wireless Ethernet) may be coupled toone or more input/output devices, such as a keyboard, a pointing device,a scanner, or a networking device such as a switch or router, e.g.,through a network adapter.

The computing device 1400 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as astandard server 1420, or multiple times in a group of such servers. Inaddition, it may be implemented in a personal computer such as a laptopcomputer 1422. It may also be implemented as part of a rack serversystem 1424. Alternatively, components from the computing device 1400may be combined with other components in a mobile device (not shown),such as a mobile computing device 1450. Each of such devices may containone or more of the computing device 1400 and the mobile computing device1450, and an entire system may be made up of multiple computing devicescommunicating with each other.

The mobile computing device 1450 includes a processor 1452, a memory1464, an input/output device such as a display 1454, a communicationinterface 1466, and a transceiver 1468, among other components. Themobile computing device 1450 may also be provided with a storage device,such as a micro-drive or other device, to provide additional storage.Each of the processor 1452, the memory 1464, the display 1454, thecommunication interface 1466, and the transceiver 1468, areinterconnected using various buses, and several of the components may bemounted on a common motherboard or in other manners as appropriate.

The processor 1452 can execute instructions within the mobile computingdevice 1450, including instructions stored in the memory 1464. Theprocessor 1452 may be implemented as a chipset of chips that includeseparate and multiple analog and digital processors. The processor 1452may provide, for example, for coordination of the other components ofthe mobile computing device 1450, such as control of user interfaces,applications run by the mobile computing device 1450, and wirelesscommunication by the mobile computing device 1450.

The processor 1452 may communicate with a user through a controlinterface 1458 and a display interface 1456 coupled to the display 1454.The display 1454 may be, for example, a TFT (Thin-Film-Transistor LiquidCrystal Display) display or an OLED (Organic Light Emitting Diode)display, or other appropriate display technology. The display interface1456 may comprise appropriate circuitry for driving the display 1454 topresent graphical and other information to a user. The control interface1458 may receive commands from a user and convert them for submission tothe processor 1452. In addition, an external interface 1462 may providecommunication with the processor 1452, so as to enable near areacommunication of the mobile computing device 1450 with other devices.The external interface 1462 may provide, for example, for wiredcommunication in some implementations, or for wireless communication inother implementations, and multiple interfaces may also be used.

The memory 1464 stores information within the mobile computing device1450. The memory 1464 can be implemented as one or more of acomputer-readable medium or media, a volatile memory unit or units, or anon-volatile memory unit or units. An expansion memory 1474 may also beprovided and connected to the mobile computing device 1450 through anexpansion interface 1472, which may include, for example, a SIMM (SingleIn Line Memory Module) card interface. The expansion memory 1474 mayprovide extra storage space for the mobile computing device 1450, or mayalso store applications or other information for the mobile computingdevice 1450. Specifically, the expansion memory 1474 may includeinstructions to carry out or supplement the processes described above,and may include secure information also. Thus, for example, theexpansion memory 1474 may be provide as a security module for the mobilecomputing device 1450, and may be programmed with instructions thatpermit secure use of the mobile computing device 1450. In addition,secure applications may be provided via the SIMM cards, along withadditional information, such as placing identifying information on theSIMM card in a non-hackable manner.

The memory may include, for example, flash memory and/or NVRAM memory(non-volatile random access memory), as discussed below. In someimplementations, instructions are stored in an information carrier. Theinstructions, when executed by one or more processing devices (forexample, processor 1452), perform one or more methods, such as thosedescribed above. The instructions can also be stored by one or morestorage devices, such as one or more computer- or machine-readablemediums (for example, the memory 1464, the expansion memory 1474, ormemory on the processor 1452). In some implementations, the instructionscan be received in a propagated signal, for example, over thetransceiver 1468 or the external interface 1462.

The mobile computing device 1450 may communicate wirelessly through thecommunication interface 1466, which may include digital signalprocessing circuitry where necessary. The communication interface 1466may provide for communications under various modes or protocols, such asGSM voice calls (Global System for Mobile communications), SMS (ShortMessage Service), EMS (Enhanced Messaging Service), or MMS messaging(Multimedia Messaging Service), CDMA (code division multiple access),TDMA (time division multiple access), PDC (Personal Digital Cellular),WCDMA (Wideband Code Division Multiple Access), CDMA2000, or GPRS(General Packet Radio Service), among others. Such communication mayoccur, for example, through the transceiver 1468 using aradio-frequency. In addition, short-range communication may occur, suchas using a Bluetooth®, Wi-Fi™, or other such transceiver (not shown). Inaddition, a GPS (Global Positioning System) receiver module 1470 mayprovide additional navigation- and location-related wireless data to themobile computing device 1450, which may be used as appropriate byapplications running on the mobile computing device 1450.

The mobile computing device 1450 may also communicate audibly using anaudio codec 1460, which may receive spoken information from a user andconvert it to usable digital information. The audio codec 1460 maylikewise generate audible sound for a user, such as through a speaker,e.g., in a handset of the mobile computing device 1450. Such sound mayinclude sound from voice telephone calls, may include recorded sound(e.g., voice messages, music files, etc.) and may also include soundgenerated by applications operating on the mobile computing device 1450.

The mobile computing device 1450 may be implemented in a number ofdifferent forms, as shown in the figure. For example, it may beimplemented as a cellular telephone 1480. It may also be implemented aspart of a smart-phone 1482, personal digital assistant, or other similarmobile device.

Various implementations of the systems and techniques described here canbe realized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations thereof.These various implementations can include implementation in one or morecomputer programs that are executable and/or interpretable on aprogrammable system including at least one programmable processor, whichmay be special or general purpose, coupled to receive data andinstructions from, and to transmit data and instructions to, a storagesystem, at least one input device, and at least one output device.

These computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor, and can be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. As used herein, the terms machine-readable medium andcomputer-readable medium refer to any computer program product,apparatus and/or device (e.g., magnetic discs, optical disks, memory,Programmable Logic Devices (PLDs)) used to provide machine instructionsand/or data to a programmable processor, including a machine-readablemedium that receives machine instructions as a machine-readable signal.The term machine-readable signal refers to any signal used to providemachine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniquesdescribed here can be implemented on a computer having a display device(e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor)for displaying information to the user and a keyboard and a pointingdevice (e.g., a mouse or a trackball) by which the user can provideinput to the computer. Other kinds of devices can be used to provide forinteraction with a user as well; for example, feedback provided to theuser can be any form of sensory feedback (e.g., visual feedback,auditory feedback, or tactile feedback); and input from the user can bereceived in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in acomputing system that includes a back end component (e.g., as a dataserver), or that includes a middleware component (e.g., an applicationserver), or that includes a front end component (e.g., a client computerhaving a graphical user interface or a Web browser through which a usercan interact with an implementation of the systems and techniquesdescribed here), or any combination of such back end, middleware, orfront end components. The components of the system can be interconnectedby any form or medium of digital data communication (e.g., acommunication network). Examples of communication networks include alocal area network (LAN), a wide area network (WAN), and the Internet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

Elements of different implementations described herein may be combinedto form other implementations not specifically set forth above. Elementsmay be left out of the processes, computer programs, databases, etc.described herein without adversely affecting their operation. Inaddition, the logic flows depicted in the figures do not require theparticular order shown, or sequential order, to achieve desirableresults. Various separate elements may be combined into one or moreindividual elements to perform the functions described herein. In viewof the structure, functions and apparatus of the systems and methodsdescribed here, in some implementations.

Throughout the description, where apparatus and systems are described ashaving, including, or comprising specific components, or where processesand methods are described as having, including, or comprising specificsteps, it is contemplated that, additionally, there are apparatus, andsystems of the present invention that consist essentially of, or consistof, the recited components, and that there are processes and methodsaccording to the present invention that consist essentially of, orconsist of, the recited processing steps.

It should be understood that the order of steps or order for performingcertain action is immaterial so long as the invention remains operable.Moreover, two or more steps or actions may be conducted simultaneously.

While apparatus, systems, and methods have been particularly shown anddescribed with reference to specific preferred embodiments, it should beunderstood by those skilled in the art that various changes in form anddetail may be made therein without departing from the spirit and scopeof the invention as defined by the appended claims.

What is claimed is:
 1. A system for registration of one or more 3-Dimages of a subject, the system comprising: a processor; and a memorywith instructions stored thereon, wherein the instructions, whenexecuted by the processor, cause the processor to: (a) receive a 3-Dimage of the subject, wherein the 3-D image comprises a graphicalrepresentation of one or more bones of interest; (b) identify in thegraphical representation one or more bones of interest and a referenceobject; (c) automatically generate one or more seed artificial objects,each seed artificial object being within a first distance interval fromthe reference object and corresponding to a region of a bone of interestin the graphical representation; (d) for each automatically generatedseed artificial object, automatically generate a plurality of associatedsubsequent artificial objects, thereby creating a set of artificialobjects within the image; (e) automatically perform registration of oneor more images of the subject using the set of artificial objects withinthe image; (f) identify one or more dangerous regions of the imagecorresponding to regions in which a distance between a first and secondbone of interest in the graphical representation is below a predefinedthreshold distance; and (g) automatically generate each artificialobject such that it is sufficiently far from any identified dangerousregion within the image.
 2. The system of claim 1, wherein theinstructions cause the processor to automatically generating the one ormore seed artificial objects by: determining a distance map comprisingintensity values at each of a plurality of points in three dimensions,each of the intensity values of the distance map corresponding to adistance from a given point in 3-D space to the reference object;generating a distance interval mask from the distance map; and applyingan AND operation between the distance interval mask and a maskcorresponding to the identified one or more bones of interest, therebyidentifying a plurality of regions of the identified bones of interestthat are within the first distance interval from the reference object.3. The system of claim 1, wherein for each automatically generated seedartificial object, the plurality of associated subsequent artificialobjects comprises an equidistant chain of artificial objects.
 4. Thesystem of claim 1, wherein each artificial object is a predefinedthreshold distance from an identified additional bone within thegraphical representation.
 5. The system of claim 1, wherein theinstructions cause the processor to automatically generate the pluralityof subsequent artificial objects associated with a respective seedartificial object by generating a chain of artificial objects along abone of interest by beginning with the seed artificial object and, in astepwise fashion, generating new subsequent artificial objects, eachnewly generated artificial object proceeding outwards, away from theidentified reference object, along the bone of interest.
 6. The systemof claim 5, wherein the instructions cause the processor to generate thechain of artificial objects associated with the respective seedartificial object by, for at least one newly generated artificial objectof the chain of artificial objects: determining whether the newlygenerated artificial object is within a predetermined threshold distancefrom an identified additional bone in the graphical representation; andresponsive to determining that the newly generated artificial object iswithin the predetermined threshold distance from the identifiedadditional bone of the image, terminating generation of subsequentartificial objects associated with the respective seed artificialobject.
 7. The system of claim 1, wherein each artificial object of theset of artificial objects within the image is confirmed to have at leasta predefined threshold volume.
 8. The system of claim 1, wherein theinstructions cause the processor to: determine, based on the set ofartificial objects within the image, an image registrationtransformation; and apply the image registration transformation to aregion of the 3-D image, thereby registering the 3-D image.
 9. Thesystem of claim 8, wherein the image registration transformation yieldssymmetrization of the 3-D image, thereby correcting distortions in theimage.
 10. The system of claim 8, wherein the received image correspondsto a first image, and the image registration transformation aligns thefirst image with a second image of the subject, thereby co-registeringthe first image with the second image.
 11. The system of claim 8,wherein the instructions cause the processor to determine the imageregistration transformation by: determining, from the set of artificialobjects within the image, a set of artificial landmarks within theimage, each landmark corresponding to a point determined from acorresponding artificial object; and determining the image registrationtransformation using the set of artificial landmarks within the imageand a set of target landmarks, wherein the image registrationtransformation is determined to, when applied to points corresponding tothe artificial landmarks within the image, substantially optimizealignment of the set of artificial landmarks with the set of targetlandmarks.
 12. The system of claim 11, wherein the set of targetlandmarks is symmetric.
 13. The system of claim 11, wherein theinstructions cause the processor to determine the set of targetlandmarks using the set of artificial landmarks within the image. 14.The system of claim 11, wherein the set of target landmarks is a set ofpredetermined target landmarks.
 15. The system of claim 8, wherein thereceived 3-D image comprises one or more regions corresponding tographical representations of soft tissue, and the instructions cause theprocessor to apply the image registration transformation to the one ormore regions of the image corresponding to graphical representations ofsoft tissue, thereby registering the soft tissue regions.
 16. The systemof claim 8, wherein the received 3-D image of the subject corresponds toa first image recorded via a first modality, and, wherein theinstructions cause the processor to: receive a second image recorded viaa second modality; determine, based on the set of artificial objectswithin the image, a first image registration transformation; determine,based on the first image registration transform, a second imageregistration transformation; and apply the second image registrationtransformation to a region of the second image.
 17. The system of claim16, wherein the second image is recorded at substantially the same timeas the first image, with the subject in a substantially similar poseand/or position.
 18. The system of claim 16, wherein the second imageregistration transformation is the same as the first image registrationtransformation.
 19. The system of claim 16, wherein coordinates of aplurality of points of the first image are related to coordinates of aplurality of points of the second image via a known functionalrelationship.
 20. The system of claim 1, wherein: the 3-D imagecomprises a graphical representation of rib bones and a backbone, theidentified one or more bones of interest comprise rib bones, and theidentified reference object is a backbone of the subject.
 21. The systemof claim 20, wherein: the graphical representation of rib bones includesa plurality of pairs of opposite rib bones, each rib bone artificialobject of a portion of the plurality of rib bone artificial objectsbelongs to one of a plurality of pairs of rib bone artificial objects,each pair comprising a first rib bone artificial object associated witha first rib bone of a pair of opposite rib bones and a second rib boneartificial object associated with a second rib bone of the pair ofopposite rib bones, and for each pair of rib bone artificial objects,(i) a difference between a volume represented by the first object of thepair and a volume represented by the second object of the pair is belowa predefined threshold volume difference, and/or (ii) a differencebetween a height of the first object of the pair and a height of thesecond object of the pair is below a predefined threshold heightdifference.
 22. The system of claim 20, wherein the rib bones in thegraphical representation includes one or more pairs of opposite ribbones, and wherein the instruction cause the processor to automaticallygenerate a plurality of seed rib bone artificial objects by: identifyinga set of prospective seed rib bone artificial objects corresponding to aplurality of regions of the rib bones that are within a distanceinterval from the backbone; automatically identifying one or more pairsof prospective seed rib bone artificial objects, each pair comprising afirst seed rib bone artificial object of a first rib bone of the pair ofrib bones and corresponding second seed rib bone artificial object ofthe opposite rib bone; applying a pair comparison filter to the set ofprospective seed rib bone artificial objects, wherein application of thepair comparison filter eliminates, from the set, pairs of artificialobjects for which (i) a difference between a volume represented by thefirst object of the pair and a volume represented by the second objectof the pair is above a predefined threshold volume difference, and/or(ii) a difference between a height of the first object of the pair and aheight of the second object of the pair is above a predefined thresholdheight difference; and following application of the pair comparisonfilter, selecting each seed rib bone artificial object from the set ofprospective seed rib bone artificial objects.
 23. The system of claim 1,wherein the 3-D image comprises a graphical representation of abreastbone of the subject and a lower end of the breastbone.
 24. Thesystem of claim 23, wherein the one or more bones of interestcomprise(s) the breastbone in the graphical representation and thereference object comprises a lower end of the breastbone.
 25. A systemfor registration of one or more 3-D images of a subject, the systemcomprising: a processor; and a memory with instructions stored thereon,wherein the instructions, when executed by the processor, cause theprocessor to: (a) receive a 3-D image of the subject, wherein the 3-Dimage comprises a graphical representation of one or more bones ofinterest; (b) identify in the graphical representation one or more bonesof interest and a reference object; (c) automatically generate one ormore seed artificial objects, each seed artificial object being within afirst distance interval from the reference object and corresponding to aregion of a bone of interest in the graphical representation; (d) foreach automatically generated seed artificial object, automaticallygenerate a plurality of associated subsequent artificial objects,thereby creating a set of artificial objects within the image; (e)automatically perform registration of one or more images of the subjectusing the set of artificial objects within the image; (f) determine,based on the set of artificial objects within the image, an imageregistration transformation by: determining, from the set of artificialobjects within the image, a set of artificial landmarks within theimage, each landmark corresponding to a point determined from acorresponding artificial object; and determining the image registrationtransformation using the set of artificial landmarks within the imageand a set of symmetric target landmarks, wherein the image registrationtransformation is determined to, when applied to points corresponding tothe artificial landmarks within the image, substantially optimizealignment of the set of artificial landmarks with the set of symmetrictarget landmarks; (g) apply the image registration transformation to aregion of the 3-D image, thereby registering the 3-D image; (h) identifyone or more dangerous regions of the image corresponding to regions inwhich a distance between a first and second bone of interest in thegraphical representation is below a predefined threshold distance; and(i) automatically generate each artificial object such that it issufficiently far from any identified dangerous region within the image.26. A system for registration of one or more 3-D images of a subject,the system comprising: a processor; and a memory with instructionsstored thereon, wherein the instructions, when executed by theprocessor, cause the processor to: (a) receive a 3-D image of thesubject, wherein the 3-D image comprises a graphical representation ofone or more bones of interest; (b) identify in the graphicalrepresentation one or more bones of interest and a reference object; (c)automatically generate one or more seed artificial objects, each seedartificial object being within a first distance interval from thereference object and corresponding to a region of a bone of interest inthe graphical representation; (d) for each automatically generated seedartificial object, automatically generate a plurality of associatedsubsequent artificial objects, thereby creating a set of artificialobjects within the image; (e) automatically perform registration of oneor more images of the subject using the set of artificial objects withinthe image; (f) determine, based on the set of artificial objects withinthe image, an image registration transformation; (g) apply the imageregistration transformation to a region of the 3-D image, therebyregistering the 3-D image that corresponds to a first image recorded viaa first modality; (h) receive a second image recorded via a secondmodality; (i) determine, based on the set of artificial objects withinthe image, a first image registration transformation; (j) determine,based on the first image registration transform, a second imageregistration transformation; (k) apply the second image registrationtransformation to a region of the second image; (l) identify one or moredangerous regions of the image corresponding to regions in which adistance between a first and second bone of interest in the graphicalrepresentation is below a predefined threshold distance; and (m)automatically generate each artificial object such that it issufficiently far from any identified dangerous region within the image,wherein the second image is recorded at substantially the same time asthe first image, with the subject in a substantially similar pose and/orposition.